Automated Videoing Assistant - Made with TensorFlow.js

[@VALedu11]

This is fantastic. I am sure including multiple person detection in it would be an addition. More so, localizing on one person of all those are detected in every frame. Wish you more such hardware based innovations.

[@automatedvideoingassistant6192]

Yes, it has multiple person detection. It will detect any 'person' in the field of view. The end result is quite random - switching from one person (following them) then switching to another person and following them instead. It's similar to how a human would film a group of people (focusing on one person then another), but AVA can't of course pick out the human 'in view' doing the most interesting thing to focus on. It will typically flip to the human form that gives the highest inference score.

[@automatedvideoingassistant6192]

If we can get the Kickstarter off the ground, we can't wait to see what the community of JS and other devs can make AVA do. Handpose, Facemesh, voice commands are all super exciting areas with many great use cases.

[@JulianAlimin]

Can you use it to Track the Ball instead of the Person? Was thinking of applications in Football or Basketball.

[@automatedvideoingassistant6192]

We are looking at the 'Sports Ball' class in the MobileNet v2 model and getting interesting results. Currently tracking of the ball is good-ish - currently some challenges with significant ball speed and vertical height. Vertical height is a problem, poor AVA can't turn her head upwards when the ball goes out the top of the field of view and she struggles to understand which way to look :-)

[@JulianAlimin]

@@automatedvideoingassistant6192 that is encouraging. Cool then

[@ruman2494]

Such a cool implementation.

[@ben6]

This is a limited version of what we can buy already (I purchased from amazon for £50). A gimbal can do this tracking, plus multi access rotations, stabilisation, and more features. This is still a great achievement, to manufacture the hardware and write the software. Very impressive.

[@automatedvideoingassistant6192]

Yes, there are a number of 3-axis Gimbals with auto-tracking, but they're designed for slow movement smooth following. We have a few for testing and they're poor at tracking anything with speed and usually only tracking a person/face. We can track anything in the MobileNet v2 class or use other ML models. PIVO is our closest match, but there's nothing out there that will give you - the developer, the opportunity to make code your own great projects out of AVA.

[@ayaanp]

That's really cool!

[@pouyahallaj4445]

That's really cool! Would be cooler if you can reduce the jitteriness!

[@automatedvideoingassistant6192]

Hi Pouya, yes, this was our original prototype and had a few important firmware omissions and lessons that we learned. We fixed this on the latest prototype which is quick and smooth. If you're a dev you can also code onto the robot too IF we can get our current Kickstarter backed.

[@pouyahallaj4445]

@@automatedvideoingassistant6192 Thanks for your reply, I'm very happy to hear that! Hopefully you could get your Kickstarter backed. I'm also an Developer/Videographer and can see a future for this device!

[@automatedvideoingassistant6192]

@@pouyahallaj4445 many thanks. In theory, with handpose, facemesh..etc, face scrunching and hand gestures could move the device, so we're excited too. We're just over 10% of our funding goal and 5 days into the campaign. Hope to get one of these to you one day.

[@pouyahallaj4445]

@@automatedvideoingassistant6192 I would happily support this if i could, unfortunately I'm in Iran and i we can't have any international transactions, because of the sanctions :D but you should keep up the good work!

[@PritishMishra]

That's cool man...

[@kurisutofusan]

I wanted to build something like that but I have no idea how to interface the servo motors with the Bluetooth module... 😞

[@aricooper-davis3605]

Take a look at microcontroller boards like arduinos. They're very well documented and beginner friendly, with lots of libraries to make driving servos and Bluetooth modules easier. Some even have Bluetooth chips built in. You can code in C or in micro python these days.

[@kurisutofusan]

@@aricooper-davis3605 thanks! Arduino was on my "to check" list already. That makes it even more interesting and I'll be sure to try it!

[@automatedvideoingassistant6192]

@@kurisutofusan Yes, Arduino is a great place to start. You can add on sensors, easily get github code and get a great development environment for your hardware idea.

[@alamjim6117]

Well, thanks for showing a real project rather than trying to explain theoritical topics

[@thankyouthankyou1172]

I have thought about drone-like Automated Videoing Assistant. it can follow people automatically.

[@shashankkumar851]

What is the research opportunity in tensor flow.js

[@JasonMayes]

Can you elaborate? There are many areas you can do research for your field of study and use tfjs to assist that. We are already seeing folk from HCI research community try TFJS for their research too or existing folk who use our Python version convert Py models to run in web so more people can try their research and get feedback to then improve the model further etc

[@mehtabtv6671]

Nice

[@akshayakumars2814]

TensorFlow 🤙

[@lastsoldier8102]

Hey guys...nice..! You watched Eagle Eye movie?😁