#PMChallenge – Collection and Curation Hub

  1. knodalyte Cupcake Feb 8, 2019

    knodalyte, Feb 8, 2019 :

    The Android environment includes rudimentary functions that enable a user to "share" snippets of text, URLs, images, files, and other artifacts created, consumed, or captured by the user. The system of intents and the ability to "share" from many applications provides a foundation, but a weak user experience, limited configurability, and general lack of attention by the Android team leave significant gaps that OxygenOS can fill.

    The goal of this enhancement is to use the power of intents and the share function to make the Oneplus phone the central point in a user's digital experience. To implement this, Oneplus developers need to implement:
    - the ability to capture almost anything visible on the screen in a form that is most usable to the user, including text, image, or link
    - the option to be able to share captured artifacts immediately or later on
    - capability for doing further processing of artifacts, including editing text, cropping and annotating images, and merging artifacts
    - improvements to the sharing experience to enable the user to understand and manage share targets to meet personal needs and preferences

    Who are the users?

    Many users would benefit from improving the sharing UI and providing a configuration UI to customize sharing options and provide high-level guidance about the Android intents system capabilities.

    For users who want to have a more powerful way to capture and re-use text and images on their Oneplus phones, the Collection and Curation Hub would elevate them to another level by making easy to manage digital assets while leveraging capabilities built into Android apps.

    What is the proposed function?

    There are three main workflows that this proposal depends on:

    Configuration of sharing would be a new UI in the system settings. It would include some introductory information about some of the types of explicit intents that Android supports and shows a list of applications that can be targets for each intent type. It would be even better if it were possible to show which applications can originate each intent type, but I don't know if that is possible. Within each list of target apps, the user would be able to rank which apps should appear at the top of the list (or even be the default app) and also which apps should be hidden when a share intent is triggered.

    The main enhancement in this area would be creating an interface to select an area of the screen that should be captured. This capability exists in somewhat limited form by the existing text selection method of dragging over text in an app, triggering the clipboard popup. This mechanism is somewhat awkward, requiring some fidgeting to select the specific text of interest. In addition, once text has been selected, the actions available are limited, although having the option to share the text is likely to suffice in many cases.

    The bigger shortcomings of current Android functionality are the lack of support for even basic text selection in some applications and the inability to select any non-text content. By adding the ability to capture any rectangle of screen content and then handle it either as an image or apply OCR to it to reduce it to text, we can overcome current limitations and make digital content more available for sharing and collection.

    The capture UI would be triggered by executing a gesture or by interaction with a persistent screen overlay object. When the screen content has been marked, then a popup overlay would offer options for further actions, including sharing, copying to the clipboard, or searching.

    By including a sharing capability in the capture UI, we sidestep inconsistencies in sharing functionality in the originating application, and invoke the system share handler that is appropriate for the content type.

    See the link to the fooView application below for an example of a working capture function.

    The third workflow provides new functionality to the Android environment. One or more new applications - ideally a single application for handling the most common intent types, alternatively distinct apps for each of the common intent types - would be available as targets for share intents. When triggered, the app would display the content that had been shared (text, URL, image, filename, etc.) and would allow the user to perform actions on the content, or simply save the content for possible use at a future time.

    Any type of content could be further processed by sharing it to applications that can handle that type of intent.

    Alternatively, the Curation app would include tools to enhance content. For example, text content could be edited, multiple content items could be merged and then shared, or content could be deleted. Image content could be cropped or processed by OCR (which would result in a new content item of type text).

    The Curation app could optionally be configured to function as a text clipboard aggregator.

    The Curation app would allow the user to save any content item as a file, with options to set the filename of a text file to match the first line of text.

    See the links to the SupportText and Clipboard Manager Pro apps below for examples of curation capabilities.

    What is the user value?

    At a minimum, implementing these suggestions would make the Android sharing capabilities more accessible and consistent (see Android's sharing system is morphing into an unpredictable mess | Computerworld).

    For user who do a significant portion of their information gathering on their smartphones, and for users who want better tools for sharing content on social media, the Capture and Curation interfaces enable precise and powerful marshalling of digital content. This kind of power further marginalizes technologies like PCs and laptops as well as competing ecosystems like iOS.

    If there are existing comparable functions in the smartphone industry, how is your proposed feature superior?

    The Play Store has applications that provide comparable functions, but they have limitations and shortcomings. This kind of functionality should be rolled into the OS where it can be implemented in a consistent, clear, integrated, and attractive way.

    fooView - Float Viewer - Apps on Google Play does a pretty good job of capturing content in an efficient way; it also provides some clipboard management capability but not as much as the Curation tool that is proposed here. The app includes many functions and most users would be too confused to configure and use it. It is a pretty amazing app and a work in progress.

    SupportText Pro - Apps on Google Play is another lightweight yet very powerful tool, with almost nonexistent documentation (unless you read Japanese). For text processing in the small, it is very interesting, but again configuring and exploiting it are beyond most users without a huge documentation and UI upgrade.

    Similar to SupportText, there are several clipboard management apps that offer persistence, editing, tagging and searching as well as multiple sharing options. One typical one is Clipboard Manager Pro - Apps on Google Play but like SupportText, it is oriented solely toward text. Image and other content type are missing.

    !Share, Andmade Share, Sharedr, AutoShare, Share Selector and a few other apps attempt to bring a bit of order and configurability to the share menu, with varying degrees of success. Again, this should part of OS settings configuration.

    Ruby G. likes this.
  2. Sridhar Ananthanarayanan Lollipop Feb 9, 2019

    Sridhar Ananthanarayanan, Feb 9, 2019 :
    Can you give a couple of real-life use cases where this would be superior to existing tools?

    1. We have the option to capture screenshots of what we see on the screen, with the ability to edit it before sharing.

    2. Everytime we choose a text or a string of texts, Google predicts and provides ("smart select") an appropriate next action choice. If it is a phone number, we see the phone app, if it is an address, we see Maps, etc.

    3. Clipboard - yes that would be welcome. But I would prefer one that is available on multiple platforms and can sync. For example, Microsoft is introducing it. I don't know why Google hasn't done it so far.

    Would be interesting to see how your suggestion would have a better real life application.

    alessiodales likes this.
  3. knodalyte Cupcake Feb 11, 2019

    knodalyte, Feb 11, 2019 :
    Unless I want to capture the full screen including the status bar, it takes more steps to end up with a small part of the screen by using a screen capture and crop. Why not directly select the area of interest? It's about efficiency.

    I find the Android text selection interface to be clumsy at best. Sometimes it takes several attempts to capture just what I want. Easier to select a block of text and edit it, maybe add a tag or other annotation, then pass it to the next step using clipboard or share options. It would be nice to have more control over my options beyond the list of shares, for example setting up a custom search query URL or REST call.

    I am less interested in sharing a clipboard to another device, although there many times when that would be very handy. I tend to want to share files (for which I use SyncThing).
    Last edited: Feb 11, 2019