25
Deep Dive: Inside The OnePlus Phone Piano

  1. Ard
    Social Staff Member Nov 26, 2019

    Ard , Nov 26, 2019 :
    Piano-ForumPost-(1).png

    Hi friends,

    There’s been much talk about our OnePlus Phone Piano recently, so let’s take this opportunity to delve a little deeper into this rather incredible piece of equipment.

    Just like an ordinary piano, our remarkable instrument produces not only memorable notes but, even more impressively, plays beautiful music – as seen and heard during our recent European tour.

    Except, the Phone Piano is no ordinary piano. In fact, it’s pretty extraordinary – thanks to the technical wizardry lying beneath the piano lid.

    A Real, Life-Size Piano Minus Traditional Keys

    The Phone Piano is identical to a traditional piano in every way – except for the keys. Instead, 17 OnePlus 7T Pro handsets are arranged in a line to form a seamless and smooth keyboard. Each handset is programmed to display a set of black and white keys that are fully playable.

    The handsets sit in custom routed trays cut using computer-controlled tools from a solid block of fiberboard. This allows the phones to sit incredibly close to one another while maintaining position both horizontally and vertically.

    As for the rest of the piano, one of the main considerations was portability. Having to travel across Europe meant the equipment had to be light, robust and durable, as well as being able to fold neatly into two flight cases. So, a suitably light whitewood was chosen, and the design finalized in 3D modeling application Cinema 4D.

    The Science Of Sound


    Inside the OnePlus Phone Piano sit two computers. The first is a server that receives signals and the second is a synthesizer to produce accurate notes. Alongside these is a series of speakers and a subwoofer to amplify the sound. The bottom of the piano has specifically crafted acoustic ports to allow the soundwaves to flow optimally.

    We, of course, couldn’t have brought any of this to life without our OnePlus 7T Pro flagship – all 17 of them. As any pianist will tell you, playing the instrument requires immediate feedback as soon as you touch the keys. Luckily, ultra-responsiveness is something our 90 Hz Fluid Display technology excels in.

    The Software Running The Show


    Each handset has its own unique custom application installed. This talks to both the main server and the accompanying handsets. A keystroke sends a signal to the server, which is converted to a MIDI signal that is sent to the synthesizer to play the corresponding note.

    Like most Android apps, the primary phone software used is written in Java, which allows for flexibility when developing within the Android ecosystem. Since we knew there would be 17 handsets all forming a central interface, the app features a set-up function allowing the team to program which octave, key, and port each individual handset is using.

    When a key is pressed, the phone app sends a ‘key press event’ to the central server via TCP connection types. On the server side, the software (written in NodeJS) receives the ‘key press event’. It then translates every key press and its duration into a MIDI signal. This MIDI signal exits the server via a MIDI cable into the synthesizer which produces the note in an analog form and sends it to the speakers housed inside the piano.

    Making Smartphones Look And Feel Like Keys


    To ensure our pianist, gifted musician Karim Kamar, knows which keys are being pressed, a touch-sensitive graphical user interface was created that displays a glowing effect – depending on how long the keys are pressed.

    During the construction phase, the key images and scaling needed to be adjusted to both fit the handset while also still being playable at the same time. The exceptional responsiveness of the 90 Hz Fluid Display allows for smoother animations, with no lag between pressing the keys and the visual representations.

    Engineers were able to reduce to three milliseconds the time it takes from touching a key to the sound being audible using a special TCP connection. Once this was confirmed, additional handset functionality such as note range was added.

    For Karim to be able to sustain a note – as traditional pianos can do – our engineers created a special pedal. By reverse engineering a simple PC gaming pedal, we were able to link this back directly to the server to provide sustain modulation within the MIDI signal for Karim to use.

    If you are curious to find out more about the OnePlus Phone Piano, read more in our announcement and interview.
     

    #1
  2. obakesan
    Jelly Bean Nov 26, 2019

    obakesan , Nov 26, 2019 :
    For people who didn't see here's some behind piano scenes quick footage I shot at the day in London. A look inside the Piano .

     

    #2
  3. camohan
    Marshmallow Moderator Nov 26, 2019

    camohan , Nov 26, 2019 :
    The more I see it, more I feel this was really really an interesting concept.
    I want to learn playing piano and have tried some very very basics. I can feel how difficult it can be to play on small instrument like mobile device.
    Great concept, great precision.
     

    #3
  4. obakesan
    Jelly Bean Nov 26, 2019

    obakesan , via OnePlus 7 Pro , Nov 26, 2019 :
    Thanks looking everyone, a few more photos, as much I have sadly, hopefully others might have more from the other cities or even London.

    As @Ard said, you can see the speakers connected here and might see those laptops too. It's amazing how it all fits inside hidden away, quite a unique device and really fascinating, well done to the oneplus team!


    1574791265166.jpg

    1574791265282.jpg
     

    #4
  5. obakesan
    Jelly Bean Nov 26, 2019

    obakesan , via OnePlus 7 Pro , Nov 26, 2019 :
    My one regret is I couldn't play Piano! I did a few notes but would have been awesome to play it properly
     

    #5
  6. camohan
    Marshmallow Moderator Nov 26, 2019

    camohan , Nov 26, 2019 :
    It's worth preserving in OnePlus HQ.
     

    #6
    manuel19, jlasensiofi and obakesan like this.
  7. camohan
    Marshmallow Moderator Nov 26, 2019

    camohan , Nov 26, 2019 :
    I can understand it could be extremely difficult to make your mind and fingers work together to such unusual setup.
     

    #7
    manuel19, jlasensiofi and obakesan like this.
  8. obakesan
    Jelly Bean Nov 26, 2019


    #8
    manuel19 and jlasensiofi like this.
  9. obakesan
    Jelly Bean Nov 26, 2019

    obakesan , via OnePlus 7 Pro , Nov 26, 2019 :
    Ya the one thing I noticed speaking with Karim was there is no depth like you get on a usual piano. The keys are flat and no give take so that itself makes it a challenge for a seasoned pro, an amateur like me hard enough haha
     

    #9
    jlasensiofi likes this.
  10. camohan
    Marshmallow Moderator Nov 26, 2019

    camohan , Nov 26, 2019 :
    Yes the HQ reception lobby is quiet big to accommodate the piano. Even if not the reception, the game room has ample space, staff could occasionally come and play for sometime as a recreational activity.
     

    #10
    jlasensiofi and obakesan like this.
  11. FlixbusLennart
    The Lab - OnePlus 6 Reviewer Nov 27, 2019

    FlixbusLennart , via OnePlus 7 Pro , Nov 27, 2019 :
    Hey @Ard, thanks for the deep dive! Another entry to the series that deserves the name.

    How did you charge the phones? Running a whole day, displaying the white keyboard buttons, must be quite stressful for the batteries? How did you make sure the devices would stay charged, without wireless charging?
     

    #11
    jlasensiofi likes this.

  12. #12
    Mradul_Bhatt likes this.
  13. Impersonal
    Jelly Bean Nov 27, 2019

    Impersonal , Nov 27, 2019 :
    When you dismantle the piano, you should giveaway the phones used for this strange/ wierd contraption....

    It would also atract more than 10 comments ;)
     
    Last edited: Nov 27, 2019

    #13
    jlasensiofi and obakesan like this.

  14. #14
  15. Ard
    Social Staff Member Nov 27, 2019

    Ard , Nov 27, 2019 :
    Good question! We charged them in advance, and they lasted from the moment we started playing until we finished without a problem.
     

    #15

  16. #16
    obakesan likes this.
  17. michelin
    Gingerbread Nov 27, 2019

    michelin , Nov 27, 2019 :
    [e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e][e]1f631[/e]
     

    #17
  18. michelin
    Gingerbread Nov 27, 2019


    #18
  19. cloudtalahari
    Eclair Nov 28, 2019

    cloudtalahari , Nov 28, 2019 :
    So cool!
     

    #19
    obakesan likes this.
  20. GuitarHero0179
    Jelly Bean Nov 28, 2019

    GuitarHero0179 , Nov 28, 2019 :
    I think the latency for audio is really something that can be improved on Android comparing it to the Apple side.

    Looking at pro-audio almost all manufacturers stay away from Android integration as the latency is said to be too high. This is a cool concept though :)
     

    #20