Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
Sounds like a good idea. But to reduce costs for server and stuff I would think about a decentralized network of devices which notify each other. I dont know whether you wanted to use neural nets or sth like this. If you do so you also could teach one device a sound and its meaning/priority and due to the decentralized network only one device has to "know" it. Anyways to reduce network traffic you also could synchronize them onto each device... I would love to help
-
@hyvte Well first server would be my PC, only cost is electricity.
I'm thinking the parts will be designed to implement API/interfaces so they can be swapped out/replaced as it gets more advanced.
The server also hosts the sound recognition algorithms so I don't think decentralized will work. Need a "powerful" machine for that, it's will be like the brains of the system.
I'm probably gonna be doing an MVP/POC first but yes I don't know any ML so yes help would be great.
Although first gotta figure out how to get the audio to the server, and some way to splice I think based on the amplitude? i.e. actually detecting a sound vs just background noise/silence -
@billgates Maybe making it possible to run client & server on the same device would solve this problem?
Anyways you could analyse the amplitude on the "client" and only then send the sound to the server. This would save a lot of traffic. Especially outdoor where you have no WiFi connection it would save a lot of Highspeed volume.
Tag me when you need help.
I would first develop the algorithm and see whether it really needs a server or if it also could run on a client device... -
@billgates Ah then this issue is solved. But outdoor would make sense too. E.g. cars honking
-
@billgates I didnt mean as a driver :D As a pedestrian... I dont know because you said you dont wanna go outside because you cant hear such things
-
@billgates ah okay sorry. Then I misunderstood you :/ But indoor wouldn't it be easy to modify your door bell and things like this to give an optical Signal? Okay that would cost much more I noticed. Do you want to use your smartphone as input? You carry it maybe always with you and static microphones only optional?
-
@hyvte $$$$$$$$
Whole reason I tried to get Google to do it... I even wrote about it.
https://medium.com/@c41feb25d8dd/...
They got the parts... Just need to hack it together... but now I gotta do it myself... -
@billgates As said before I would like to help you. This were only first thoughts which helped me to understand your idea.
-
@dontPanic still thinking about design... Didn't get far today tho... N too tired now... Maybe this weekend...
Yes guess I should do the server first and just feed it some dummy data... Figure out how to do ML on the audio... -
@optravox thanks, for GUI I'm thinking bootstrap + React. Write once, run anywhere.
But prolly will be the last part. Need to get the server and processing logic down first.
Not really sure how I will do it at this point.
I just need "something that can be trained to recognize specific audio patterns even with background"
Need to think about that and do some learning and build some MVPs first -
@billgates sounds dope I'll go check some things on web after this cool beer ;)
-
Concerning the trainsmition of the audio.
I've looked into it a bit and it seems either sending amplitude data or spectograms as images is recommended.
Also google did an experiment about sound classification. https://aiexperiments.withgoogle.com/...
It has some similarities with your project.
The most important words for you should be T-SNE and librosa.
Short summary of the terms:
T-SNE is a clustering algorithm for high dimensional data (like audio)
Librosa a python library by Google, that helps you analizinng and visualizing audio data.
Would love to help you with this project, but I'm still not confident enough with tensorflow and ML in general.
Hope this helps 😉 -
@Awlex thanks! neither am i, need to learn all this stuff.
Was planning to start looking into it this weekend but life happened.... Things with higher priorities.... :( -
@Awlex just checked out the demo, wow this is pretty much what I need.
But looks like the project code is not as complete as their demo, just drums. So trying to reach out to the guy now see if can get more info on how to capture and convert the sounds -
@andyk thanks but not quite what I'm looking for. Speech recognition has already been solved.
-
@Awlex fck so just spent all night re-setting up my Windows Python env... but can't get these libs to work..... Seems like I need to go with Linux but... I don't have the space for the VM.... n not sure if Cygwin or Win 10 Ubuntu shell can handle it... Would have to install Python n Java again too....
don't have enough space... And I ain't switching my main OS from Windows to Linux... -
@billgates Since I've gone through this (for too long) I understand how you feel. It's possible to setup everything in windows.
For environment:
Add
<Path to python> and <Path to python>\Scripts to the systemvariable path.
For Python: Make sure your Pythonversion is 3.5.x (IMPORTANT!) since Tensorflow only supports that. You can test that with "Python --version"
For Tensorflow:
Make sure to get the most out of your hardware by installing tensorflow-gpu. There will be Errors about not being able to install dependencies. You'll have to manually download precompiled libraries for windows. (You can get them here: www.lfd.uci.edu/~gohlke/pythonlibs)
Choose the versions that match your environment and install them with pip.
In the install docs tensorflow tells you which stuff you have to install too use your graphic card.
If you have any other questions feel free to ask -
@Awlex oh Tension Flow is Python too? Guess I need to change Java to Python.
The problem I had actually was trying to get the AI Experiments app working.... It does look like it has most of the parts needed but... can't get it to work properly in Windows.... -
@billgates Can you specify what you can't work on Windows? Maybe I can help
-
@mohammed I tried setting up the demo
https://aiexperiments.withgoogle.com/...
And the underlying I think.
https://github.com/kylemcdonald/...
But that needs some SampleRates library not available on Windows.
Also seems Anaconda and Jupyter Notebook isn't working but I think it's a PATH issue... Jupyter can't "import utils" but Anaconda in the installer says you it's not recommended to hard code the Python location in PATH -
@-eth you mean there's already an app or it could be another output device?
I guess if it gets to Production that could make me a lot of $$$... hm... But still need a 24/7 listening device... that could freak ppl out just like Google Glass... -
Alright so got some audio samples and the Python audio analysis thingy sorta working.... Didn't need to get the demo working or install Linux.,
Just needed Librosa and actually reading the ReadMe.md....
Now to build my sound library and see how to classify all these doorbells and other sounds with TensorFlow
-
@billgates just foznd this, might be interesting for your project :
https://reddit.com/r/raspberry_pi/... -
@-eth this is what I was thinking. These are all separate IoT projects. Ideally, one wants a doorbell or fire alarm to send a signal to a wearable to vibrate a code. While capturing sound, like a doorbell, is a great idea, its eminently more practical to send a signal to a wearable.
-
@localghost no idea, that's what I need to learn... But haven't gotten around to it yet these few weeks. I was thinking TensorFlow does it.
-
@coretool but there are only specific sounds needed?
Like for doorbell just record the doorbell ringing a few times and feed it to the system.
I guess in ML terms, the main problem is classification, assuming you can extract/clean the sound from a almost continuous audio stream? -
@coretool Basically I'm thinking of something like "OK Google", listening 24/7 and in IFTTT terms:
-If hears a sound
-Then classify it based on tagged sound samples/training data -
@billgates IMHO keras with tensorflow backend it's easier to use compared to tensorflow itself, it has a great learning curve, you not need to build the layers but there are some preset as Dense (fully connected layer), Convolution2D (also 1D and 3D for convolutional layers), Flatten, Dropout, MaxPooling, these are the common used and you can build an NN in few lines of python, you can save the model and the weights or use tensorboard in just one line of code and finally there is a nice project to load your trained NN in browser with keras-js :)
-
@coretool does nothing... Or it uploads it to the internet and crowdsources an Answer... Though given how successful Tay had been Im a bit skeptical that will work...
-
@coretool thanks, at the moment i m still kinda thinking on idea space. Not too familiar with ML actually at least not the current kind. Took a basic Data Mining course years ago but nothing keep. None of the Forier transform, algo stuff just how to use DM software like Weka.
Any books you'd recommend to get started with some practical hands-on knowledge, get up to speed (ie. not academic theoretical textbook BS) and related to what I'd like to do? -
@billgates look at Machine Learning in Action, great light book that's not a dry read but also teaches a lot
-
@Axis but aren't all the... In Action long and dry like a reference manual?
I think I tried Scala In Action before but gave up pretty quick... -
@billgates I cannot speak for any of the other in action books but basically this one provided a little math lesson (linear algebra and probability) and every chapter afterwards is a different ml technique that is taught with an example that you are supposed to follow along with. It isn't the most fun book ever but i wouldn't say it is dry
-
Yo easy on that shiny new word "Tensorflow". You can't jus use that to solve any ML problems. Training a deep neural network is very costly and it's not necessary in every problems. Several lightweight ML frameworks are there.
https://blogs.nvidia.com/blog/2017/...
Btw the idea sounds great. Let's collaborate. Python/Node/Android/Firebase - Anything and I am in. -
Python : collect the training samples for the needed sounds .
Train a classification model with audio features like MFCC .
Classification algorithms : depends on the data volume .. SVM would work well with less data. could also try random forest and xgboost .
Put the model on raspberrypi with a running loop recording audio for some given time and then further classifying it as music ,car honks ,people talking ,etc usimg the trained classification model
Flash the output on screen( classified label.) -
1. there already are solutions like this precisely to help deaf people (sorry, can't remember specific names)
2. from what i've read from the devs, the sound recognition is easy
3. what is extremely hard is to make/train the system to successfully filter out background noise. has to be trained/tailored for every single location's soundscape specifically, where even each room in a house is considered a separate location (meaning currently it's impossible to have a mobile phone be the microphone, even in ideal conditions) -
@billgates
1. sorry, can't find the specific thing i came across back then even though it was the only one that looked most advanced and most similar to what you are talking about. it was a tech video explaining how it works, they mentioned the name only in passing, i'll look some more, but for now, maybe useless, but i found things like this: http://lssproducts.com/category/...
3. visual pairing might help, at least for cars? set up a camera with a directional microphone pointed at the road, to a) be able to kinda learn how the soundwaves of passing cars look, b) be able to cross-reference these recordings with background noise stuff in you main microphone, the one you're trying to clean up background noise from?
there's a lot of discussion in here already and i haven't read it, so i'm sorry if i'm just repeating stuff that was already discussed -
@billgates
#1 got it! i remembered it was a computerphile video about the topic, i recommend watching it, even though it's on the border of "layman explanations", but there's still things that are interesting&relevant&informative even for us hardcore techies ;)
https://youtube.com/watch/...
guy talking in the video is CEO of the company working on this stuff, their page is
https://audioanalytic.com/people/...
there's also some more interesting/possibly useful links in the video description. hope this helps at least a bit -
there's also another approach you could take at filtering relevant/irrelevant sounds (to a degree), based largely where they're coming from (outside vs. inside). look at this, sound reconstruction from video (visual microphone):
http://news.mit.edu/2014/...
https://youtube.com/watch/...
the software they use is opensource.
now imagine: two visual microphones, one pointed at an exterior window, another pointed at "bag of chips" (receptive surface) inside, what's picked up by the window pane but not the "bag of chips" is from outside, what's picked up by both (or just the chips) is inside, first rough filter pass done. but maybe this just adds an unnecessary middleman, as the same can probably be done with normal microphones and I don't know if it adds any advantages... -
@billgates yes, i noticed the "on hold" part, but I bet it doesn't stop you from still thinking about it, and... that's what I'm kinda doing and good at - providing some new food for thought (hopefully) and inspiration ;)
A sound recognition system
Being deaf is one major factor that's kept me from moving out. I can't hear sounds like doorbells, fire alarms, etc
I want to build a client-server solution that can be trained to recognize specific sounds
A sound recognition system<br /> <br /> Being deaf is one major factor that's kept me from moving out. I can't hear sounds like doorbells, fire alarms, etc<br /> <br /> I want to build a client-server solution that can be trained to recognize specific sounds