Singing is about so much more than just hitting the right notes. Breath management is one of the most important skills you can develop to improve your singing. Proper vocal technique helps you avoid injuring your voice when tackling difficult songs. Learning to recognize and use techniques like belting, twang, head voice, chest voice, vocal fry, etc., can really bring your singing to life. Prosody and rhythm training help you develop good timing (one of the most important aspects of music), and there's so much more to it.
This kind of app is great for getting started and getting a taste of what singing involves, but it would be awesome to cover these other aspects as well, or at least mention their existence for aspiring singers.
I'm not great at any of these things. I do agree with you... do you have links to good resources in this regard? Maybe I can start by linking to them, and then later building the features that make sense into the app.
I am sure this app will be very useful for my wife and daughter, who spent the whole day singing at the top of their lungs. It will also be a benefit for me, as I’ll get to hear more accurate notes.
Ok, I'll be that one today. This seems entirely vibe coded... why is it using webview/tauri? It could be so small and smooth in swift with just a small change to the prompt. People used electron/tauri/whatever to make things easier to write and maintain, but if we're just vibing things anyway, can we go back to respecting resources, please?
I like the idea though. I was using a paid app doing something similar in the past. Keen to get back into it.
You're not wrong. I do have a coding background which helps but I'm new to Rust. It started in Python with a web frontend so it somewhat evolved into this.
> why is it using webview/tauri?
Using these makes it easier to port to the other platforms later if I choose to.
About 365 of the 420 MB is the AI models which I decided to bundle so that the app wouldn't have to download additional things after the install.
My agent tells me that writing in native Swift would save (estimate) about 100–170 MB, out of about 1.4 GB when the models are active. And then I'd have to maintain the interface in two or three ways if I decided to make versions for other platforms.
If it does grow though and there are a lot of Mac users, this might be worth it though as you suggest. But I have a lot of project that go precisely nowhere so I always keep this in mind :)
What a fun idea! I love that the song doesn't have to be programmed in by the developer. Will give it a try.
Have you played the Rock Band and similar "rhythm games" that were all the rage in the Wii era? I don't know that it helped my singing per se but gave some incentive to practice.
In case anyone is interested, you can test your relative pitch perception (as opposed to producing the pitch) in this excellent Yale site: https://www.themusiclab.org/quizzes/td I got 15th [sic] percentile!
Didn't know that external apps can interact with Youtube in a tab in Safari.
Looks nice, I sing so badly that I never dare to try if not alone. Maybe this app will help me to improve just little that I can sing along happy birthday quietrly in a corner without anyone turning heads.
Yes, this was me. My confidence has improved through practise to a point where it only half killed me to post that demo video. My family still ask me to relocate when I'm doing this around them... so I think you can hit the pitches and still not sound amazing.
Unfortunately, the way it works, and to avoid any copyright issues, it can't really operate on phones because you can't listen to other apps on them. Maybe the only way is to share your learnt pitch tracks (trained on the laptop) with a mobile companion app.
I think my daughter will love this. How about some simple gamification mechanism? A score or a star rating on how well you performed, maybe combined with a difficulty level?
Awesome, I'm checking it out right now. Just a few quick notes:
- By “difficulty,” I meant that you can set a skill level that's a bit more forgiving if you miss the mark, as opposed to a “pro” level, which may be stricter. I’m thinking especially of kids, who might have more fun with it that way. It would also be nice to have a personal profile to choose from, with your own library (?) or at least your own presets (difficulty, progress, etc.).
- It took me a little while to figure out how the app works. You start a song in Apple Music, and the app records it. I had somehow expected a large search bar in the app that I could use aside from the music suggestions to get started.
- Even though I hadn’t set up Spotify (it wasn’t even displayed), it was still enabled in the settings.
- This is probably a bit outside your scope, but in the german song “An Tagen wie diesen, Die Toten Hosen” some (final) syllables get cut off, and sometimes the bars go down even though the melody goes up (or are the notes transposed an octave when they no longer fit within the scale?).
That's for now, I’ll keep an eye on the project and create issues if my observations become more consistent. Thanks for the project!
> By “difficulty,” I meant that you can set a skill level that's a bit more forgiving
Got it! If anything, it's on beginner level right now and is very forgiving and I need to add maybe two levels above it with pro being at the top. Let me know if you think it needs something even more forgiving than how it works currently and I can try to build that in as a 4th level.
> I had somehow expected a large search bar in the app that I could use aside from the music suggestions to get started.
Yeah, the app relies on you finding the music in the way you normally do in your music players, I'll try and make this clearer. Once you have a few songs in your library however, then you can use that to navigate through them or search them up by difficulty and play them.
> Even though I hadn’t set up Spotify (it wasn’t even displayed), it was still enabled in the settings.
Thanks, will check that out.
> or are the notes transposed an octave when they no longer fit within the scale?
Do you mean this... Now and then a note bar is drawn an octave off its neighbours. The cause is the note model (GAME) choosing the wrong octave for that note. It happens most on pop with heavy backing, where a backing vocal or instrument sits an octave away. I've tried to fix, but the evals get worse so I've not managed to fix that issue yet.
There are definitely some issues I haven't been able to solve yet and it creates terrible pitch tracks for some songs. I have an eval set that I'm optimising through changes so hopefully it gets better over time. It might be at the limits of what's possible with the vocal separation, but I'm hoping that it can get better than it is right now with time and effort.
If you yourself sing an octave above or below, it will draw adjust and score you as on key.
> That's for now, I’ll keep an eye on the project and create issues if my observations become more consistent.
PS: There seems to be a bug on my machine. When I press the Play button in the Library, the song screen opens, but the previous song performance is still open there and nothing happens. I have to press the "in Apple Music" in the library; then it switches to Apple Music, where I press Play again and switch back to OnKey. Can't OnKey start playback on its own?
For the user's pitch OnKey uses CREPE-tiny (https://github.com/marl/crepe) but ported to Rust with the model running on ONNX Runtime. For the singer's pitch also CREPE but the full version.
Singing is about so much more than just hitting the right notes. Breath management is one of the most important skills you can develop to improve your singing. Proper vocal technique helps you avoid injuring your voice when tackling difficult songs. Learning to recognize and use techniques like belting, twang, head voice, chest voice, vocal fry, etc., can really bring your singing to life. Prosody and rhythm training help you develop good timing (one of the most important aspects of music), and there's so much more to it.
This kind of app is great for getting started and getting a taste of what singing involves, but it would be awesome to cover these other aspects as well, or at least mention their existence for aspiring singers.
I'm not great at any of these things. I do agree with you... do you have links to good resources in this regard? Maybe I can start by linking to them, and then later building the features that make sense into the app.
I am sure this app will be very useful for my wife and daughter, who spent the whole day singing at the top of their lungs. It will also be a benefit for me, as I’ll get to hear more accurate notes.
Great. Let me know your feedback here, in the GitHub as an issue, or on Discord!
Ok, I'll be that one today. This seems entirely vibe coded... why is it using webview/tauri? It could be so small and smooth in swift with just a small change to the prompt. People used electron/tauri/whatever to make things easier to write and maintain, but if we're just vibing things anyway, can we go back to respecting resources, please?
I like the idea though. I was using a paid app doing something similar in the past. Keen to get back into it.
You're not wrong. I do have a coding background which helps but I'm new to Rust. It started in Python with a web frontend so it somewhat evolved into this.
> why is it using webview/tauri?
Using these makes it easier to port to the other platforms later if I choose to.
About 365 of the 420 MB is the AI models which I decided to bundle so that the app wouldn't have to download additional things after the install.
My agent tells me that writing in native Swift would save (estimate) about 100–170 MB, out of about 1.4 GB when the models are active. And then I'd have to maintain the interface in two or three ways if I decided to make versions for other platforms.
If it does grow though and there are a lot of Mac users, this might be worth it though as you suggest. But I have a lot of project that go precisely nowhere so I always keep this in mind :)
The main demo video uses the Red Hot Chilli Peppers as an example. A singing app, for learning to sing, and Anthony Kiedis used as the model.
Red Hot Chilli Peppers. Anthony Kiedis.
It's like if you had let Shaquille O'Neal teach you how to shoot threes.
Yes, I got this from a friend... "Lol, I could t tell the difference between you and Tony Kiedis!"
I'm terrible at detecting sarcasm usually, but this one was easy for me.
I now realize this comment may read much more mean-spirted than I intended--not my intention!
This is a great and cool app. I am just joking. Sorry to be an ass.
Not at all! It's accurate :)
I'd definitely still let Shaquille O'Neal show me how to shoot threes. Dunking on the other hand could be an issue.
What a fun idea! I love that the song doesn't have to be programmed in by the developer. Will give it a try.
Have you played the Rock Band and similar "rhythm games" that were all the rage in the Wii era? I don't know that it helped my singing per se but gave some incentive to practice.
In case anyone is interested, you can test your relative pitch perception (as opposed to producing the pitch) in this excellent Yale site: https://www.themusiclab.org/quizzes/td I got 15th [sic] percentile!
Not played Rock Band myself only some Guitar Hero.
https://www.themusiclab.org looks cool, I'll give it a try today when I get a chance.
Can you exclude a section from scoring if it picked up a backing vocal instead of the melody?
Didn't know that external apps can interact with Youtube in a tab in Safari.
Looks nice, I sing so badly that I never dare to try if not alone. Maybe this app will help me to improve just little that I can sing along happy birthday quietrly in a corner without anyone turning heads.
> I never dare to try if not alone
Yes, this was me. My confidence has improved through practise to a point where it only half killed me to post that demo video. My family still ask me to relocate when I'm doing this around them... so I think you can hit the pitches and still not sound amazing.
Unfortunately, the way it works, and to avoid any copyright issues, it can't really operate on phones because you can't listen to other apps on them. Maybe the only way is to share your learnt pitch tracks (trained on the laptop) with a mobile companion app.
I think my daughter will love this. How about some simple gamification mechanism? A score or a star rating on how well you performed, maybe combined with a difficulty level?
Each song has a difficulty rating from 1 to 5. You can use this to filter your library.
And you can track your scores over the takes you did of a sing in your history. So I think OnKey has you covered on these.
Let me know if it matches what you expect when you try it out.
Awesome, I'm checking it out right now. Just a few quick notes:
- By “difficulty,” I meant that you can set a skill level that's a bit more forgiving if you miss the mark, as opposed to a “pro” level, which may be stricter. I’m thinking especially of kids, who might have more fun with it that way. It would also be nice to have a personal profile to choose from, with your own library (?) or at least your own presets (difficulty, progress, etc.).
- It took me a little while to figure out how the app works. You start a song in Apple Music, and the app records it. I had somehow expected a large search bar in the app that I could use aside from the music suggestions to get started.
- Even though I hadn’t set up Spotify (it wasn’t even displayed), it was still enabled in the settings.
- This is probably a bit outside your scope, but in the german song “An Tagen wie diesen, Die Toten Hosen” some (final) syllables get cut off, and sometimes the bars go down even though the melody goes up (or are the notes transposed an octave when they no longer fit within the scale?).
That's for now, I’ll keep an eye on the project and create issues if my observations become more consistent. Thanks for the project!
> By “difficulty,” I meant that you can set a skill level that's a bit more forgiving
Got it! If anything, it's on beginner level right now and is very forgiving and I need to add maybe two levels above it with pro being at the top. Let me know if you think it needs something even more forgiving than how it works currently and I can try to build that in as a 4th level.
> I had somehow expected a large search bar in the app that I could use aside from the music suggestions to get started.
Yeah, the app relies on you finding the music in the way you normally do in your music players, I'll try and make this clearer. Once you have a few songs in your library however, then you can use that to navigate through them or search them up by difficulty and play them.
> Even though I hadn’t set up Spotify (it wasn’t even displayed), it was still enabled in the settings.
Thanks, will check that out.
> or are the notes transposed an octave when they no longer fit within the scale?
Do you mean this... Now and then a note bar is drawn an octave off its neighbours. The cause is the note model (GAME) choosing the wrong octave for that note. It happens most on pop with heavy backing, where a backing vocal or instrument sits an octave away. I've tried to fix, but the evals get worse so I've not managed to fix that issue yet.
There are definitely some issues I haven't been able to solve yet and it creates terrible pitch tracks for some songs. I have an eval set that I'm optimising through changes so hopefully it gets better over time. It might be at the limits of what's possible with the vocal separation, but I'm hoping that it can get better than it is right now with time and effort.
If you yourself sing an octave above or below, it will draw adjust and score you as on key.
> That's for now, I’ll keep an eye on the project and create issues if my observations become more consistent.
Thank you!
PS: There seems to be a bug on my machine. When I press the Play button in the Library, the song screen opens, but the previous song performance is still open there and nothing happens. I have to press the "in Apple Music" in the library; then it switches to Apple Music, where I press Play again and switch back to OnKey. Can't OnKey start playback on its own?
This is a great learning tool! I wish I could use it on Linux. :)
Noted, I haven't done a feasibility test yet, but I do have an issue up here to vote on: https://github.com/dsmurrell/onkey/issues
It's got 3 thumbs up but no indication of which OS they're from... maybe I need two issues.
How are you detecting pitch?
For the user's pitch OnKey uses CREPE-tiny (https://github.com/marl/crepe) but ported to Rust with the model running on ONNX Runtime. For the singer's pitch also CREPE but the full version.
The vocal separation for the song uses UVR: https://github.com/Anjok07/ultimatevocalremovergui
You can find more details here: https://singonkey.app/docs/how-it-works