Calvin Wankhede / Android Authority
TL;DR
- OpenAI is testing a more efficient way to let ChatGPT see your screen on Android.
- It could use Android 17’s new Bubbles feature and Accessibility options to view what’s on your screen.
- This method uses system resources more efficiently compared to the current screen-sharing method that involves casting.
OpenAI is looking for a way out by using a new Android 17 feature that could change how screen sharing with ChatGPT works on Android.
ChatGPT on Android currently uses the MediaProjection API, which is also used for screen recording or casting the screen to an external display, to see what’s on your screen. This method can feel clunky for various reasons, including the different permissions dialogs and warning pop-ups that appear, which can obstruct your workflow. Since the API also continuously records the screen, this can lead to high resource utilization, slowing down other processes.
To address these concerns, OpenAI is developing a new method that could serve the same function while using fewer resources. In version 1.2026.118 of the ChatGPT app on Android, we’ve discovered that OpenAI is toying with the idea of using Android’s Accessibility settings, along with Android 17’s Bubbles multitasking feature, to let the chatbot see what’s on your screen.
The feature changes the workflow for Screen Sharing in ChatGPT. When it is enabled for the first time, users will be required to enable certain settings so ChatGPT can “read visible text, buttons, and screen details.” They’re first guided to enable “ChatGPT screen help” in Android’s Accessibility settings, following which they would need to enable notifications and conversation bubbles for the app. These settings would prevent the app from being killed.
Once permissions are granted, a conversation bubble will appear on your phone’s screen, and you can use it to see what’s on your screen, whether you’re on the home screen or in an app.
Don’t want to miss the best from Android Authority?
You will be able to interact with ChatGPT by tapping this bubble. Here you will have the option to ask specific questions about the screen’s contents, and it will ignore any context from the existing chat, focusing only on what it currently sees on your screen.
While this approach bypasses the excessive resource utilization of the existing screen-sharing method, granting Accessibility access to an AI chatbot could also be a concern for some wary users. We’re unsure how OpenAI plans to deal with those apprehensions, or how soon we could see the feature being implemented.
⚠️ An APK teardown helps predict features that may arrive on a service in the future based on work-in-progress code. However, it is possible that such predicted features may not make it to a public release.
Thank you for being part of our community. Read our Comment Policy before posting.
