During covid I needed to setup remote piano lessons. I tried combining two videos to show hands on the keys as well as the face but at the time merging the two videos added too much latency.
I solved this by only using the top down camera and then adding clear perspex (from a plastic picture frame) at 45 degrees in front of the raised laptop and also pointed lamps at the piano players face (the effect is called Pepper's Ghost). This meant the teacher could see the hands on the keyboard and the players face with one camera. It also meant the piano player would appear to look directly into the camera - like an auto cue.
I'm posting here in case it helps someone else.
This is such a cool concept, congrats on shipping!
I've been developing a webMIDI app myself for the past couple of years [0] and adding a multiplayer feature is something I've been eyeing for a while. May I ask how good the latency is?
Some feedback on the landing page:
- As others already pointed out, a video example showing the app would be really helpful.
- I think allowing visitors to try out the piano and only prompting to login when trying to access the remote features would work best
- All those buttons that are clickable/selectable but don't do anything when clicked were a little confusing.
On the app:
- Connecting the keyboard worked flawlessly (and without any permission prompt). But I'd add some indication that a MIDI device is connected
- Pressing the keys using my mouse doesn't produce any sounds
- Pressing keys on my MIDI keyboard (Yamaha P120) will cause the keys to be selected indefinitely, so I can only ever press them once. Somehow the noteOff events don't seem to be getting registered. Make sure you also register noteOn events with velocity = 0 as a noteOff event (although I don't believe that that's the issue here) and I would suggest calling noteOff yourself if you register a noteOn event for a note that's already playing. That way you can prevent keys from getting stuck.
- I wouldn't hide keys automatically based on screen size. It does look better but imagine someone trying to play this on a tablet/phone where they can't change the scaling
Again, really cool concept and good luck with further development! Feel free to reach out if you want any help :-)
Oh nice, another HN'er that's on the same wavelength :) I'll give your app a spin and send you some feedback, meanwhile, maybe there are useful ideas in pianojacq.com that you can adopt.
Ohh Hi! I've come across pianojacq before! Definitely love how clean the sheet music looks and on first glance the way the song plays smoothly even when you hit the note a little late/early works really well! That's one aspect I really need to improve.
I see you're also using Vexflow. Are you using a library to go from Midi to Vexflow format? It's a rabbit hole I sometimes wish I hadn't entered :)
@OP Can really recommend VexFlow [0] when you do get to implementing sheet music.
I used two separate libraries, one to convert the MIDI file into an in memory representation and Vexflow, then a whole pile of glue (not the nicest part of the code) to combine the two.
There are some aspects of that that I want to revisit such as the automatic recognition of decorations (for instance: trills) so that they can be shown properly but these are tricky problems and I want to avoid spending more time on pianojacq.com than on using it.
Lately I've added an ear training module but I'm not completely happy with it yet (hit Bb0 to enable a hidden menu if you want to play around with it).
You're both on the same midi channel too, so you'll play well together, and should compare notes and velocities, and send each other your running status.
(Where's my drum machine when I need a badoom psssssh?)
I really appreciate the feedback, Bewelge, these ideas are all very helpful to me! I'll be making changes based on each one.
I haven't measured the latency directly. But, I did try this with my sister who lives across the US while also on a phone call with her. When she speaks or plays a note, I hear it through the app sooner than over the phone. So it's at least better than that!
As a student, I don't care about a two-sided platform (which is what it looks like you're trying to build). I want to sign up for vetted piano lessons through your site, and purchase a USB midi adapter that you recommend. As a short-term MVP, perhaps point people to a known-working adapter on a site with an affiliate program. Long-term you might consider finding a company willing to sell one white-label and put your logo and website on it, but that's a lot less important than everything else you're building so put it off for a while.
Also, for the first handful of lessons and for subsequent practice, it'd be awesome if you had automated, pre-recorded lessons available, to get the basics down, or to train things like reading sheet music at speed.
Regarding "USB MIDI adapter": Actually USB MIDI is the lowest common denominator for MIDI keyboards. If you pick up any used MIDI keyboard less than 10 years old, it will have USB. Something in the $40-$60 range on Craigslist is perfect.
The big choice for the beginner (or anyone) is how many keys do you want, and have space for? It goes all the way from 25 keys up to 88.
Then going up in price, models will start to include (in random order) old 5-pin MIDI in and out, velocity sensitive keys, (semi-)weighted keys, aftertouch, inputs for foot controllers, pads/knobs/faders for controlling a DAW, built-in sequencers, external sync in/out, CV and gate outputs for interfacing with modular gear, etc..
If you are at all serious about piano I would recommend getting an 88 key keyboard with half decent keys. Anything less you'll outgrow in the first few months and you will be learning to properly orient yourself. On a smaller keyboard you're going to mess up your 'South Sight' and your 'North Sight' because you will train yourself to orient on the edges of your smaller keyboard. As soon as you sit down behind a proper piano you'll be confused.
Good keyboards aren't expensive any more, the simplest Yamaha, Casio and Roland that qualify for serious study are all around $500 if you shop around a bit and if you go for second hand can be had for half that. You'll very quickly match that in lesson costs so if you can afford the one you probably can afford the other.
You don't need sequencers, aftertouch, built in DAWs or controlling an external one etc, those are not aimed at piano students. Can be fun but complete luxury.
But it's a space where more expensive older hardware might still be worth it. E.g. my full piano size 24 year old Yamaha w/velocity sensitive keys and pedals still works flawlessly and the current version of it is still expensive new (at least from the point of view of a casual user...)
It's now connected via a cheap USB adapter to my Chromebook (biggest issue: latency of the - mostly Android - apps) planning to connect it to my Linux box instead.
You can always install Linux on any Chromebook, or install a Linux VM (but that may not make it trivial to directly attach devices). To the best of my knowledge 100% of Chromebooks are fully supported.
That already exists, and is a relatively robust market, with multiple providers. I used https://pianomarvel.com/ for a while during COVID with some success.
I'd love to give some feedback both as a teacher and a student. But I don't see a way on your website to just see what it really does and how it works, beyond just those couple of small animations.
At least a YouTube demo video, or ideally direct access to a real demo interface would be more helpful.
I thought Getting Started might help, but that forced me into a sign-up flow that I don't want to invest in without more information about the outcome.
Thank you for the response. You're right, it didn't sit well with me putting a required log in between the landing page and the app. So I'll work on a more comprehensive video demo and make it accessible before signing up.
From what I've heard MIDI controllers are not good for learning piano. You really want weighted keys and a full 88 keys otherwise you won't learn the right habits or build finger strength.
(I learned on a physical piano in person, so I can't speak personally about learning on a MIDI controller, but it sounds plausible to me.)
It will be quite a while before anything else besides the student just engaging with playing and sticking with it matters at all. IMO.
To that end, I think a crappy plastic MIDI keyboard can do a better job than a steinway if it is easier to acquire and accommodate. Being at hand is more important than being just right at this stage.
I say this as a music major, but it is not in any way a universal opinion to be sure.
You can also get midi interfaces for cathedral organs; letting you use the organ console to control midi devices, and also letting midi devices control the pipe relays in the pipe organ.
Of course, whether you would want to use the OP's app in some giant cathedral is another question.
I agree that weighted keys are preferable, but it's not about "building finger strength," but rather about building finger coordination. Your fingers are already plenty strong enough for weighted keys (especially once you learn how to take advantage of gravity in your technique).
But weighted keys are crucial for building coordination. The resistance of the keys helps you calibrate the connection between your touch and the sound produced. You want both kinesthetic and aural feedback.
The thing is, all pianos feel different. It's one of the downsides of the instrument is that you often end up having to play whatever piano is at the event (barring the aforementioned digital pianos that are portable).
If all you have is a digital keyboard, don't let that stop you for learning it!
Strength is an issue, but most of that is 'the only good musician has been dead for at least 100 years'. A good keyboard plays just as well as a geal piano. You can feel the difference so we can't do a real ABX test, which means unlike audio gear we cannot do objective tests.
organs have always had their own feel and plenty of greats have proven they sound great.
my opinion (as a dad with kids learning piano, who learned piano as a kid) playing with 88 weighted keys is like learning to write cursive with a fountain pen - maybe artistically interesting, but doesn't matter for the core skills. My kids are learning on 61 key unweighted boards and they're learning melody, rhythm, notation, theory, and all of the things that are about music, not about the particular physiological requirements that a machine originally built in the 1600s imposed.
Weighted keys aren't about requirements from the 1600s, they're about being able to achieve an extremely wide and sensitive dynamic range.
Unweighted is fine if you're making synth sounds. And of course for understanding the things you've listed.
But you can't actually play emotionally expressive piano music on them -- not in the style of classical or jazz. If you tried to play the Moonlight Sonata first movement, it would sound terrible, because the dynamic shadings couldn't be done.
I'm not really sure how you've determined what "core skills" are. Sure, if your kids are only going to spend a couple years learning the very basics, and then move onto other things, then it's fine.
But the heart and soul of piano music is in the precise touch to generate the dynamic sensitivity. If you want them to learn how to be emotionally expressive through music, playing either classical or jazz, weighted keys are essential. It's not piano otherwise -- it's a synth.
(And going from unweighted to weighted isn't trivial. It's an entirely different muscle memory that needs to be developed. They're fundamentally different instruments.)
>But you can't actually play emotionally expressive piano music on them -- not in the style of classical or jazz.
Yes you can, and yes it is more-or-less trivial. With a small amount of deliberate practice, you can learn to produce a full and finely-gradated dynamic range on either a piano-weighted or a semi-weighted keybed. It's a one-dimensional mechanical skill that just isn't particularly difficult relative to, say, an oboeist's reed control or a violinist managing the very complex relationship between bow and string. If I say "you just don't press quite as hard" it'll sound like I'm being glib, but that's literally all there is to it, because physics.
Strings players choose from a wide range of string tensions and woodwind players choose from a wide range of reed stiffnesses, based purely on their own personal preference. Some prefer something soft and pliable that responds to the lightest touch, others prefer something that fights back when you dig in. It's an accident of history that mechanical pianos fall into a relatively narrow range of weightings, not a deliberate choice on the part of pianists or piano makers - the range of options are limited by the mechanics of an escapement and hammer.
Pianists overwhelmingly prefer weighted keybeds out of habit, but more generalist keyboard players will often prefer a semi-weighted keybed for versatility. You can play gigs or sessions with either and you're the only one who's going to notice. Calling either choice wrong is just dogma.
I don't know where you're getting your information from, but it's just wrong, and you also misinterpreted my comment. Responding:
> Yes you can, and yes it is more-or-less trivial. With a small amount of deliberate practice, you can learn to produce a full and finely-gradated dynamic range on either a piano-weighted or a semi-weighted keybed. It's a one-dimensional mechanical skill that just isn't particularly difficult relative to, say, an oboeist's reed control or a violinist managing the very complex relationship between bow and string. If I say "you just don't press quite as hard" it'll sound like I'm being glib, but that's literally all there is to it, because physics.
Pretty much every piano teacher on the globe would like to have a word with you. The idea that it's a "one-dimensional mechanical skill" is greatly misleading when it has to do with joints in the fingers, wrist, elbow, shoulder, even your spine. Positions of all those things, varying degrees of muscle tension. And it's incredibly difficult -- pianists spend years improving their touch, and do things like Alexander lessons to eliminate muscle tension that interferes. "Don't press quite as hard" is incredibly complex, physiologically. (And oboeists are only playing one note at a time, violinists one or two -- pianists need to learn to independently control the force of all 10 fingers independently, sometimes all at once!)
But if you want to understand it in just a one-dimensional way, you can. Imagine that the full range of force a finger can produce is mapped from 0 to 100, and humans are sensitive to the degree they can adjust that by 1. Now imagine a weighted keyboard is responsive to the values between 10 and 60 -- that's 50 levels of sensitive gradation. An unweighted or semi-weighted keyboard is more like 25 to 35 -- merely 10 levels. Because we have so much less control, the software makes sure that the extremes are clipped -- you can't play as quietly or as loudly. Your dynamic expressiveness is severely limited. If you want to say "because physics", that's your "because physics".
You can measure this in MIDI outputs, actually. A good weighted keyboard will give you a wide range and an experienced pianist can consistently hit the same values without much noise. Whereas unweighted either gives you a much smaller range of values, or you can crank the output range up with software settings but discover that there's either a ton more signal noise because we don't have that fine control over our muscles, or else you discover that the intermediate MIDI values aren't even being used because the keyboard sensors don't support it (e.g. they only support 8 values).
> Pianists overwhelmingly prefer weighted keybeds out of habit, but more generalist keyboard players will often prefer a semi-weighted keybed for versatility. You can play gigs or sessions with either and you're the only one who's going to notice. Calling either choice wrong is just dogma.
That's false. Pianists don't prefer weighted out of habit, they prefer it because it's required for greater expressiveness, which classical/jazz requires.
The "more generalist" keyboard players you're referring to doing "gigs or sessions" are generlly not aiming for dynamic expressiveness -- they're part of a band or providing basic accompaniment to singers doing pop songs, and so forth. And that's fine -- unless it's Norah Jones-type music, you're not usually asking for much expressiveness from the piano in pop music. (She is much more jazz, after all.)
But that isn't "more generalist", it's pop. They're not practicing or performing Beethoven or Chopin, because it doesn't work except on a weighted keyboard. Nor are they doing jazz piano. You can't do it. Not well, anyways.
And you claim that "calling either choice wrong is just dogma", but that's wrong twice. First, I was very clear than there's nothing wrong with playing the synth. If you want to play synth music, I was clear that unweighted is great (you wouldn't even want weighted, actually). I specifically said weighted is necessary for classical and jazz. But second, it's not "dogma" that you need weighted for classical and jazz. It's just facts. It's "because physics", and the sensitivity of human physiology specifically. It's not dogma -- it's just reality. It's how the two instruments work.
This isn't quite correct, or at least not for the reasons you've articulated. "Weighted" versus "semi-weighted" has no such inherent limitation of ranges or values, even leaving aside that the numbers you've chosen are probably simply for the sake of your example. The actual limitation between most weighted and semi-weighted keyboards is that semi-weighted tend to have inferior sensors (and possibly other components), because these instruments tend to be marketed towards people who desire to spend less money on an instrument. There is no technical or physical limitation that requires semi-weighted keyboards to have less expressive range.
Unweighted keyboards tend to have simple springs to return the keys to resting position, and more importantly, the cheapest instruments tend to only have note-ON;note-OFF characteristics due to being unable to sense any variation in how a note is played. These instruments playing all notes at the same volume is a natural consequence of inadequate sensors.
Returning to your sample values, MIDI 1 only directly supports 256 levels of volume, I think? This 8-bit level of variation is far lower than what any normal human sensory input can be expected to discern. MIDI 2, which was still very rare when last I paid heed, is a substantial improvement for potential expressivity. But even many years later, most consumer-grade instruments and playback devices only supported MIDI 1, sometimes with extensions such as GS or XG. The most expressive of the pro or semi-pro instruments that I've experienced have only seemed fully capable when using their internal proprietary enhancements; when outputting to a MIDI 1, the results were much less impressive.
As an aside, I've played only one piano whose keys felt as lightly weighted as my first semi-weighted keyboard and I found it a delight to use. Fully expressive, and less effort for the same effect as other pianos. I felt like I could play it for hours without tiring. By comparison, keys on all other pianos I've tried now feel quite heavy. I wish I could have afforded that one at the time. Sadly, memory of its make and model are long garbage collected.
> There is no technical or physical limitation that requires semi-weighted keyboards to have less expressive range.
Of course there is -- it's not a technical limitation of the instrument, but of our physiology.
If there's more weight resisting, this gives us a wider range of control, for the reasons I explained. Of course only to a point -- once it gets too heavy that a light touch can't move it, or the heaviest touch becomes too fatiguing, you've gone too far. But pianos are weighted in a way that approximately maximizes our level of expressive control without being fatiguing.
So yes, semi-weighted keyboards inherently have less expressive range. I mean sure, not if you're a robot playing with infinitely fine motor control. But for actual human beings, semi-weighted are absolutely less expressive. There's no getting around it.
And with regards to the one extremely light-touch piano you found a joy to play on -- I'm quite sure that it couldn't have been played extremely softly in an even way. That's just the inherent tradeoff. If you're playing music than never goes to ppp then maybe you didn't notice. But I don't need to see the piano myself -- this is all just physics and physiology. It's inherent to how it works.
As a piano/keyboard player, a lot of musicality is possible on a keyboard. It is possible to learn to modify the technique to better utilise the velocity available to a particular keybed, weighted, or non-weighted. When playing keyboards you are working within a subset of the potential dynamics available to a piano. Though expressivity is lessened, there is still a huge palette once you learn to use less total force and less differentiation in force (dynamics).
I know I can play with high musicality on almost any keyboards with velocity, because I was blessed to have learned to use bad instruments. But, it doesn't compare to the depth of the sound generated by all the moving parts and interactions happening in a real piano. Not only the sounds, but also the sheer weight of the keys.
Most* keyboards/vsts are just triggering a (pitch-shifted, looped) sample at a given note and then doing that for n notes and that doing an additive sum.
That is definitely not what occurs in a piano though. There you have the 3-dimensionality of the physical world, like the way waves are traveling through distance and shape. When sounds' harmonics interact, resonant nodes in overtone sequences can trigger each other to resonance, which can trigger other resonances throughout the tone. Maybe you know the feeling of depressing the sustain/damper pedal while sitting in front of it and giving the instrument a smack (or holding down the keys you are not playing and doing it). Or running your nail or a pick over all the low notes with sustain.. like you are in a cave.
In midi/digital, there's the fact that dynamic is usually gonna be 8-bit. Just because midi did that and it made sense at the time, other keyboards and VSTs mostly follow suit. I'm surprised this gets generally passed over. Obviously there's more than 128 strengths of note in real life.
But all that said I think it's possible to learn keyboards/music theory/songs/playing on a non-weighted keyboard, but false to say that digital/non-weighted is equivalent to acoustic piano. But you only really need that for really dynamic music like Jazz, Classical, Instrumental et al. But it feels so very wrong to play that kind of music on bad keyboards.
* Roland V-Piano, and PianoTeq, as well as many I'm unaware of do in fact use physical/acoustic modeling as opposed to triggering samples, but it has not been predominant even among high-end digital instruments
Try playing Moonlight Sonata Third Movement on a synthesizer! The Third movement is one of the most incredibly intricate and brilliant piano pieces I’ve ever heard in my life!
It’s more like playing baseball with a whiffle ball or using a touchscreen over a mouse, a Segway over hiking. You can learn a lot about steering riding a bike with training wheels. But just like your kids, they’ll be physically unable to use the real thing.
It’s not hard to develop that finger strength, just need give kids the opportunity
I spent 10 years playing on a keyboard with unweighted keys and I'm rather certain it affected my desire to practice. The other large part was that my teacher did not spend much time on theory with me.
Also, when recitals came around it was a big shift to playing on a grand piano with real weighted keys, and that, coupled with the nerves of playing in front of an audience, often led to mistakes.
I think the weighted keys help you feel more physically connected to the instrument with more of your body. Unweighted keys require the same involvement as typing on a computer keyboard.
It is basically like electric guitar vs classical guitar though.
Nylon has a very different touch and tension than steel.
You could start a student on electric or acoustic guitar that wants to learn classical guitar by why would you just not get them a classical guitar instead?
Same thing with piano to me. If you want to eventually learn to play some Chopin there is no reason you would not start with weighted keys.
The modern piano was also not even popular until the late 1700s.
There are lots of controllers with weighted keys these days and plenty of entry level MIDI keyboards have excellent keyboards.
The big advantage is that you don't need to tune them twice per year and that extra budget is either savings or you can spend it on more lessons. A physical piano is nice (and even those can have 'MIDI out' if you look around for a bit or are prepared to do some DIY) but really not a must. I have both here and spend much more time on my digital just because it is far more convenient, I can practice on it at any time of day even when the kids are sleeping and the feel of the keyboard is as good as the real thing (Yamaha P515, not the cheapest but very good quality).
Any digital piano functions as a midi controller, and many of them have weighted keys. And there are a few "pure" midi controllers that have 88 weighted keys, such as the M-Audio Hammer 88 or StudioLogic SL88.
There's a middle ground between the MPK mini and a steinway. You can get a used Yamaha P-series keyboard with 88 full-size, fully-weighted keys for a few hundred dollars. It won't fit in a backpack but it also won't teach you bad habits _if your goal is to learn to play the acoustic piano_.
If your goal is to learn music theory or produce music or anything other than learn to play piano, the mpk is fine. No gatekeeping here.
i encourage everyone to get started on whatever device is most accessible. if you only have a 49 key midi keyboard without weighted keys, get started on that. just do enough to have some fun and see if this is something you'd like to continue. unless you are very serious about learning classical piano, better to build bad habits and correct them later than to allow this "88 key weighted keyboard only" gatekeeping to stop you from starting the journey
I would suggest recording the videos from the player's point of view. It adds a layer of abstraction to have to reverse the image when reading it as a student.
Unless you are also encouraging your students to learn to read a pianist from an audience's perspective.
There's also https://plugandplink.com/, which includes a hardware component to help the person learning (lights on the snake).
The app also includes preprepared videos and lessons (scales, arpeggios etc.) which interact with the hardware component.
My son's been using it with a tutor remotely and it's been working great! Not only that, but he can use the app and the snake to self-guide his practise during the rest of the week.
When the pandemic hit we needed a quick solution for my daughter's piano lessons. I built a setup with OBS, Synthesia and an old Logitec web cam I had lying around (these were impossible to get new at the time).
I transmitted a frontal video of the player from the laptop cam, a top-down view of the center section of the keyboard (the cam angle was not wide enough for the whole), and the virtual keyboard from Synthesia. I had three different layouts of these elements in OBS, that could be switched by pressing the space bar. On the audio side I sent a mix from the laptop mic for voice plus the direct audio from the e-piano.
Served us well for the time.
One constraint that improved solutions probably don't have now was that teachers could not decide on a video conferencing software, so my setup-up needed to be independent. This almost made OBS a no-brainer, because of the virtual cam support.
One thing I'd change from a marketing perspective would be that in the video loop the tutor would also have a keyboard at their hands. Because usually it is like this, that if the tutor wants you to hear your mistakes, it plays the improved part which you had wrong by putting extra emphasis on the correct part while explaining it.
The video loop, as it currently is, makes if feel YouTube-y or a formal, business-like Zoom call, while the really special thing of this very interesting platform is the connectedness of tutor and student.
This looks pretty cool! Would love to see a quick YouTube demo of the product in action. What's the biggest benefit of the MIDI capability, in practice?
Thank you! I'll definitely put together a video demo before requiring logging in. Showing live MIDI during a call sidesteps the challenge of showing and recording both yourself playing and what notes you're playing. Also, because MIDI is represented digitally, we'll be able to do things like chord/scale detection, recording and playing back, and showing live playing on sheet music.
The browser (firefox) says that allowing MIDI can have security implications - what's that all about?
ETA: I didn't see this on your website because I didn't create an account or anything. But I imagine it would pop up at some point during usage, and maybe you know more about this general popup. I've seen it before on other sites that want to use MIDI devices.
Many years back I worked on a Websocket WebMidi app (https://github.com/tanepiper/browser-band) that I would have loved to have made more concrete as a way to jam online but as usual it was a sideproject that went nowhere.
Anyone have recommendations for a good midi software instruments? I'd pay for a good piano or electric sound, and I haven't been happy with the GarageBand sounds I'm currently using. What are some good affordable options, and what's the best that money can buy?
Pianoteq by Modartt seems to be well liked, it uses physical modeling rather than samples. Ivory II by Synthology is also highly regarded. Arturia makes some very nice electric piano VSTs, and some serviceable acoustic pianos as well, but they do seem a bit over priced to me personally.
I'm always amazed by how many people use it, parents that mail me about their kids 'pianojacq time' :)
There is no tracking on the site so I quite literally have no clue how many people use it and because it also works as a download there are plenty of offline users as well.
I'll get to work on a video demo! The app has a sort of piano VSTI running in the browser, so what you hear will be reliably reproduced on the peer side.
Most (if not all) digital keyboards also support receiving MIDI which will be played by the piano's own sound engine!
This will probably require assigning the received MIDI to a different channel than 0 to allow simultaneous play without conflicting note offs and sustains.
Wrote it myself. It's a javascript object that loads in all the piano sounds and manages the audiocontext. When the MIDI note/velocity/pedal state updates, a method on the object is called to play the sound.
It's meant for laptop or desktop at this time, but I will 100% look into adapting it for an iPad. I think the challenge might be connecting a midi device to it though.
Keyboard Connect is still in an early stage, so we don't support tutors advertising on the platform yet. Thank you for these suggestions!
Piano tutors who want to give an engaging remote lesson will want the ability to show themselves playing as well as what notes they're playing. This requires expensive equipment and technical knowledge to set up using Skype/Zoom. I give a few typical setups on the landing page and compare it to Keyboard Connect, which requires just a MIDI controller and webcam.
This represents a challenge for tutors and could be discouraging for new pianists who just want to try lessons without committing to buying a digital/acoustic piano are supporting AV equipment.
Also, I'll add modules to Keyboard Connect that support tasks tutors/students would normally have to organize on their own (lesson planning, file sharing, practice tools)
Because of CPU limitations, or? OBS can combine many sources at once without too much delay, or at least you can keep them in sync with the sound.
I've been developing a webMIDI app myself for the past couple of years [0] and adding a multiplayer feature is something I've been eyeing for a while. May I ask how good the latency is?
Some feedback on the landing page:
- As others already pointed out, a video example showing the app would be really helpful.
- I think allowing visitors to try out the piano and only prompting to login when trying to access the remote features would work best
- All those buttons that are clickable/selectable but don't do anything when clicked were a little confusing.
On the app: - Connecting the keyboard worked flawlessly (and without any permission prompt). But I'd add some indication that a MIDI device is connected
- Pressing the keys using my mouse doesn't produce any sounds
- Pressing keys on my MIDI keyboard (Yamaha P120) will cause the keys to be selected indefinitely, so I can only ever press them once. Somehow the noteOff events don't seem to be getting registered. Make sure you also register noteOn events with velocity = 0 as a noteOff event (although I don't believe that that's the issue here) and I would suggest calling noteOff yourself if you register a noteOn event for a note that's already playing. That way you can prevent keys from getting stuck.
- I wouldn't hide keys automatically based on screen size. It does look better but imagine someone trying to play this on a tablet/phone where they can't change the scaling
Again, really cool concept and good luck with further development! Feel free to reach out if you want any help :-)
[0] app.midiano.com
I see you're also using Vexflow. Are you using a library to go from Midi to Vexflow format? It's a rabbit hole I sometimes wish I hadn't entered :)
@OP Can really recommend VexFlow [0] when you do get to implementing sheet music.
[0] https://github.com/0xfe/vexflow
There are some aspects of that that I want to revisit such as the automatic recognition of decorations (for instance: trills) so that they can be shown properly but these are tricky problems and I want to avoid spending more time on pianojacq.com than on using it.
Lately I've added an ear training module but I'm not completely happy with it yet (hit Bb0 to enable a hidden menu if you want to play around with it).
(Where's my drum machine when I need a badoom psssssh?)
I haven't measured the latency directly. But, I did try this with my sister who lives across the US while also on a phone call with her. When she speaks or plays a note, I hear it through the app sooner than over the phone. So it's at least better than that!
A few immediate thoughts:
As a student, I don't care about a two-sided platform (which is what it looks like you're trying to build). I want to sign up for vetted piano lessons through your site, and purchase a USB midi adapter that you recommend. As a short-term MVP, perhaps point people to a known-working adapter on a site with an affiliate program. Long-term you might consider finding a company willing to sell one white-label and put your logo and website on it, but that's a lot less important than everything else you're building so put it off for a while.
Also, for the first handful of lessons and for subsequent practice, it'd be awesome if you had automated, pre-recorded lessons available, to get the basics down, or to train things like reading sheet music at speed.
The big choice for the beginner (or anyone) is how many keys do you want, and have space for? It goes all the way from 25 keys up to 88.
Then going up in price, models will start to include (in random order) old 5-pin MIDI in and out, velocity sensitive keys, (semi-)weighted keys, aftertouch, inputs for foot controllers, pads/knobs/faders for controlling a DAW, built-in sequencers, external sync in/out, CV and gate outputs for interfacing with modular gear, etc..
Good keyboards aren't expensive any more, the simplest Yamaha, Casio and Roland that qualify for serious study are all around $500 if you shop around a bit and if you go for second hand can be had for half that. You'll very quickly match that in lesson costs so if you can afford the one you probably can afford the other.
You don't need sequencers, aftertouch, built in DAWs or controlling an external one etc, those are not aimed at piano students. Can be fun but complete luxury.
It's now connected via a cheap USB adapter to my Chromebook (biggest issue: latency of the - mostly Android - apps) planning to connect it to my Linux box instead.
https://68kmla.org/bb/index.php?threads/usb-adb-converters-t...
https://www.reddit.com/r/synthesizers/comments/3wefun/midi_i...
At least a YouTube demo video, or ideally direct access to a real demo interface would be more helpful.
I thought Getting Started might help, but that forced me into a sign-up flow that I don't want to invest in without more information about the outcome.
(I learned on a physical piano in person, so I can't speak personally about learning on a MIDI controller, but it sounds plausible to me.)
All digital pianos have USB midi, and you can easily pick one a used one with weighted keys for a few hundred on craigslist.
My Yamaha U1 piano also came with a box that lets you interface over MIDI.
To that end, I think a crappy plastic MIDI keyboard can do a better job than a steinway if it is easier to acquire and accommodate. Being at hand is more important than being just right at this stage.
I say this as a music major, but it is not in any way a universal opinion to be sure.
Also, there are plenty of electric pianos, like the Wurlitzer or Rhodes, that have leas than 88 and only semi-weighted keys.
Of course, whether you would want to use the OP's app in some giant cathedral is another question.
But weighted keys are crucial for building coordination. The resistance of the keys helps you calibrate the connection between your touch and the sound produced. You want both kinesthetic and aural feedback.
If all you have is a digital keyboard, don't let that stop you for learning it!
organs have always had their own feel and plenty of greats have proven they sound great.
Unweighted is fine if you're making synth sounds. And of course for understanding the things you've listed.
But you can't actually play emotionally expressive piano music on them -- not in the style of classical or jazz. If you tried to play the Moonlight Sonata first movement, it would sound terrible, because the dynamic shadings couldn't be done.
I'm not really sure how you've determined what "core skills" are. Sure, if your kids are only going to spend a couple years learning the very basics, and then move onto other things, then it's fine.
But the heart and soul of piano music is in the precise touch to generate the dynamic sensitivity. If you want them to learn how to be emotionally expressive through music, playing either classical or jazz, weighted keys are essential. It's not piano otherwise -- it's a synth.
(And going from unweighted to weighted isn't trivial. It's an entirely different muscle memory that needs to be developed. They're fundamentally different instruments.)
Yes you can, and yes it is more-or-less trivial. With a small amount of deliberate practice, you can learn to produce a full and finely-gradated dynamic range on either a piano-weighted or a semi-weighted keybed. It's a one-dimensional mechanical skill that just isn't particularly difficult relative to, say, an oboeist's reed control or a violinist managing the very complex relationship between bow and string. If I say "you just don't press quite as hard" it'll sound like I'm being glib, but that's literally all there is to it, because physics.
Strings players choose from a wide range of string tensions and woodwind players choose from a wide range of reed stiffnesses, based purely on their own personal preference. Some prefer something soft and pliable that responds to the lightest touch, others prefer something that fights back when you dig in. It's an accident of history that mechanical pianos fall into a relatively narrow range of weightings, not a deliberate choice on the part of pianists or piano makers - the range of options are limited by the mechanics of an escapement and hammer.
Pianists overwhelmingly prefer weighted keybeds out of habit, but more generalist keyboard players will often prefer a semi-weighted keybed for versatility. You can play gigs or sessions with either and you're the only one who's going to notice. Calling either choice wrong is just dogma.
> Yes you can, and yes it is more-or-less trivial. With a small amount of deliberate practice, you can learn to produce a full and finely-gradated dynamic range on either a piano-weighted or a semi-weighted keybed. It's a one-dimensional mechanical skill that just isn't particularly difficult relative to, say, an oboeist's reed control or a violinist managing the very complex relationship between bow and string. If I say "you just don't press quite as hard" it'll sound like I'm being glib, but that's literally all there is to it, because physics.
Pretty much every piano teacher on the globe would like to have a word with you. The idea that it's a "one-dimensional mechanical skill" is greatly misleading when it has to do with joints in the fingers, wrist, elbow, shoulder, even your spine. Positions of all those things, varying degrees of muscle tension. And it's incredibly difficult -- pianists spend years improving their touch, and do things like Alexander lessons to eliminate muscle tension that interferes. "Don't press quite as hard" is incredibly complex, physiologically. (And oboeists are only playing one note at a time, violinists one or two -- pianists need to learn to independently control the force of all 10 fingers independently, sometimes all at once!)
But if you want to understand it in just a one-dimensional way, you can. Imagine that the full range of force a finger can produce is mapped from 0 to 100, and humans are sensitive to the degree they can adjust that by 1. Now imagine a weighted keyboard is responsive to the values between 10 and 60 -- that's 50 levels of sensitive gradation. An unweighted or semi-weighted keyboard is more like 25 to 35 -- merely 10 levels. Because we have so much less control, the software makes sure that the extremes are clipped -- you can't play as quietly or as loudly. Your dynamic expressiveness is severely limited. If you want to say "because physics", that's your "because physics".
You can measure this in MIDI outputs, actually. A good weighted keyboard will give you a wide range and an experienced pianist can consistently hit the same values without much noise. Whereas unweighted either gives you a much smaller range of values, or you can crank the output range up with software settings but discover that there's either a ton more signal noise because we don't have that fine control over our muscles, or else you discover that the intermediate MIDI values aren't even being used because the keyboard sensors don't support it (e.g. they only support 8 values).
> Pianists overwhelmingly prefer weighted keybeds out of habit, but more generalist keyboard players will often prefer a semi-weighted keybed for versatility. You can play gigs or sessions with either and you're the only one who's going to notice. Calling either choice wrong is just dogma.
That's false. Pianists don't prefer weighted out of habit, they prefer it because it's required for greater expressiveness, which classical/jazz requires.
The "more generalist" keyboard players you're referring to doing "gigs or sessions" are generlly not aiming for dynamic expressiveness -- they're part of a band or providing basic accompaniment to singers doing pop songs, and so forth. And that's fine -- unless it's Norah Jones-type music, you're not usually asking for much expressiveness from the piano in pop music. (She is much more jazz, after all.)
But that isn't "more generalist", it's pop. They're not practicing or performing Beethoven or Chopin, because it doesn't work except on a weighted keyboard. Nor are they doing jazz piano. You can't do it. Not well, anyways.
And you claim that "calling either choice wrong is just dogma", but that's wrong twice. First, I was very clear than there's nothing wrong with playing the synth. If you want to play synth music, I was clear that unweighted is great (you wouldn't even want weighted, actually). I specifically said weighted is necessary for classical and jazz. But second, it's not "dogma" that you need weighted for classical and jazz. It's just facts. It's "because physics", and the sensitivity of human physiology specifically. It's not dogma -- it's just reality. It's how the two instruments work.
Unweighted keyboards tend to have simple springs to return the keys to resting position, and more importantly, the cheapest instruments tend to only have note-ON;note-OFF characteristics due to being unable to sense any variation in how a note is played. These instruments playing all notes at the same volume is a natural consequence of inadequate sensors.
Returning to your sample values, MIDI 1 only directly supports 256 levels of volume, I think? This 8-bit level of variation is far lower than what any normal human sensory input can be expected to discern. MIDI 2, which was still very rare when last I paid heed, is a substantial improvement for potential expressivity. But even many years later, most consumer-grade instruments and playback devices only supported MIDI 1, sometimes with extensions such as GS or XG. The most expressive of the pro or semi-pro instruments that I've experienced have only seemed fully capable when using their internal proprietary enhancements; when outputting to a MIDI 1, the results were much less impressive.
As an aside, I've played only one piano whose keys felt as lightly weighted as my first semi-weighted keyboard and I found it a delight to use. Fully expressive, and less effort for the same effect as other pianos. I felt like I could play it for hours without tiring. By comparison, keys on all other pianos I've tried now feel quite heavy. I wish I could have afforded that one at the time. Sadly, memory of its make and model are long garbage collected.
Of course there is -- it's not a technical limitation of the instrument, but of our physiology.
If there's more weight resisting, this gives us a wider range of control, for the reasons I explained. Of course only to a point -- once it gets too heavy that a light touch can't move it, or the heaviest touch becomes too fatiguing, you've gone too far. But pianos are weighted in a way that approximately maximizes our level of expressive control without being fatiguing.
So yes, semi-weighted keyboards inherently have less expressive range. I mean sure, not if you're a robot playing with infinitely fine motor control. But for actual human beings, semi-weighted are absolutely less expressive. There's no getting around it.
And with regards to the one extremely light-touch piano you found a joy to play on -- I'm quite sure that it couldn't have been played extremely softly in an even way. That's just the inherent tradeoff. If you're playing music than never goes to ppp then maybe you didn't notice. But I don't need to see the piano myself -- this is all just physics and physiology. It's inherent to how it works.
I know I can play with high musicality on almost any keyboards with velocity, because I was blessed to have learned to use bad instruments. But, it doesn't compare to the depth of the sound generated by all the moving parts and interactions happening in a real piano. Not only the sounds, but also the sheer weight of the keys.
Most* keyboards/vsts are just triggering a (pitch-shifted, looped) sample at a given note and then doing that for n notes and that doing an additive sum.
That is definitely not what occurs in a piano though. There you have the 3-dimensionality of the physical world, like the way waves are traveling through distance and shape. When sounds' harmonics interact, resonant nodes in overtone sequences can trigger each other to resonance, which can trigger other resonances throughout the tone. Maybe you know the feeling of depressing the sustain/damper pedal while sitting in front of it and giving the instrument a smack (or holding down the keys you are not playing and doing it). Or running your nail or a pick over all the low notes with sustain.. like you are in a cave.
In midi/digital, there's the fact that dynamic is usually gonna be 8-bit. Just because midi did that and it made sense at the time, other keyboards and VSTs mostly follow suit. I'm surprised this gets generally passed over. Obviously there's more than 128 strengths of note in real life.
But all that said I think it's possible to learn keyboards/music theory/songs/playing on a non-weighted keyboard, but false to say that digital/non-weighted is equivalent to acoustic piano. But you only really need that for really dynamic music like Jazz, Classical, Instrumental et al. But it feels so very wrong to play that kind of music on bad keyboards.
* Roland V-Piano, and PianoTeq, as well as many I'm unaware of do in fact use physical/acoustic modeling as opposed to triggering samples, but it has not been predominant even among high-end digital instruments
It’s not hard to develop that finger strength, just need give kids the opportunity
Also, when recitals came around it was a big shift to playing on a grand piano with real weighted keys, and that, coupled with the nerves of playing in front of an audience, often led to mistakes.
I think the weighted keys help you feel more physically connected to the instrument with more of your body. Unweighted keys require the same involvement as typing on a computer keyboard.
…and who’d have any fun with that?
Nylon has a very different touch and tension than steel.
You could start a student on electric or acoustic guitar that wants to learn classical guitar by why would you just not get them a classical guitar instead?
Same thing with piano to me. If you want to eventually learn to play some Chopin there is no reason you would not start with weighted keys.
The modern piano was also not even popular until the late 1700s.
The big advantage is that you don't need to tune them twice per year and that extra budget is either savings or you can spend it on more lessons. A physical piano is nice (and even those can have 'MIDI out' if you look around for a bit or are prepared to do some DIY) but really not a must. I have both here and spend much more time on my digital just because it is far more convenient, I can practice on it at any time of day even when the kids are sleeping and the feel of the keyboard is as good as the real thing (Yamaha P515, not the cheapest but very good quality).
And what comes out of the speakers is the only thing that matters.
Playing the piano means different things to different people.
It just demonstrates the way in which people gatekeep music.
If your goal is to learn music theory or produce music or anything other than learn to play piano, the mpk is fine. No gatekeeping here.
I would suggest recording the videos from the player's point of view. It adds a layer of abstraction to have to reverse the image when reading it as a student.
Unless you are also encouraging your students to learn to read a pianist from an audience's perspective.
The app also includes preprepared videos and lessons (scales, arpeggios etc.) which interact with the hardware component.
My son's been using it with a tutor remotely and it's been working great! Not only that, but he can use the app and the snake to self-guide his practise during the rest of the week.
And cost is probably due to first gen, yeah. Plus, hardware development is pricey, I think.
I transmitted a frontal video of the player from the laptop cam, a top-down view of the center section of the keyboard (the cam angle was not wide enough for the whole), and the virtual keyboard from Synthesia. I had three different layouts of these elements in OBS, that could be switched by pressing the space bar. On the audio side I sent a mix from the laptop mic for voice plus the direct audio from the e-piano.
Served us well for the time.
One constraint that improved solutions probably don't have now was that teachers could not decide on a video conferencing software, so my setup-up needed to be independent. This almost made OBS a no-brainer, because of the virtual cam support.
The video loop, as it currently is, makes if feel YouTube-y or a formal, business-like Zoom call, while the really special thing of this very interesting platform is the connectedness of tutor and student.
Awesome platform!
ETA: I didn't see this on your website because I didn't create an account or anything. But I imagine it would pop up at some point during usage, and maybe you know more about this general popup. I've seen it before on other sites that want to use MIDI devices.
Can't remember if is there any piano available in the free version of NI Komplete[2] but it's still worth checking out.
[1]https://www.spitfireaudio.com/bbc-symphony-orchestra-discove...
[2]https://www.native-instruments.com/en/products/komplete/bund...
There is no tracking on the site so I quite literally have no clue how many people use it and because it also works as a download there are plenty of offline users as well.
This will probably require assigning the received MIDI to a different channel than 0 to allow simultaneous play without conflicting note offs and sustains.
Does it work on an iPad?
Keyboard Connect is still in an early stage, so we don't support tutors advertising on the platform yet. Thank you for these suggestions!
I'm happy to help test if you need a hand.
This represents a challenge for tutors and could be discouraging for new pianists who just want to try lessons without committing to buying a digital/acoustic piano are supporting AV equipment.
Also, I'll add modules to Keyboard Connect that support tasks tutors/students would normally have to organize on their own (lesson planning, file sharing, practice tools)