Android smartphone as Super-peripheral for a microcontroller

[bkf11]

Summary: Imagine if you could access all of the resources of a smartphone with a simple serial connection to your microcontroller. I need your help so that we can create an open-source app to make this possible.


Currently all sorts of electronics are getting cheaper and more powerful all the time and this is true for microcontroller-connected electronics as well. We can buy separate LCD screens, GPS, GSM, sensors, communications, data storage etc that can interface with microcontrollers. Unfortunately, the breakout boards, serial interfaces and lower volume/ specialist applications of these add on boards mean that they still cost a lot more than they could (in $ and time) and we need to learn a new interface and commands with each new add-on module. Just the breakout boards can cost more than the functional modules themselves and add bulk and complexity to a project.

I think that we should, instead, use cellphones and smartphones as integrated, compact, commodity priced, high power peripherals for our microcontrollers.

What I'd want is for a microcontroller to be able to use the resources that modern smartphones currently have. I'd want the micro to be able to receive sensor data (like GPS, accelerometers, compass, physical and on-screen buttons, digital camera, microphone, voice recognition, tilt sensor, proximity sensor, ambient light sensor etc), use the various communications capabilities built into phones (GSM voice & data, SMS, caller ID, wifi, bluetooth, NFC), able to use the computing and storage of the phone (eg flash memory, ram, floating point calculations, MP3/ video/ photo playback, signal/data processing) and the phones output devices (High resolution colour LCD screen, speaker, vibrating motor, white LED) and power supply (rechargeable battery, charging circuit). Taken all together, lower cost new smartphones are great value for money (less than $100 brand new, off contract here in New Zealand, probably even cheaper elsewhere). That's great value considering all of the things you get with it and as costs go down, and with older or damaged phones already in the second hand market, the costs can become even lower or free. A smartphone with a broken screen or other single fault can still be recycled to perform most other tasks.

Apps already exist that allow some of these functions. You can output NMEA GPS data via Bluetooth (several apps), use terminal apps to display things on the screen (eg Slick USB 2 Serial Terminal), access/see all sensor raw data on your screen (Data Monitor), use the headphone jack as a serial output (Audio Serial out), input data from micros and then graph, display or log them (BluePicaxe), output commands to your micro manually or using sensors (Arduino Commander). These efforts need to be united and expanded (or replicated and superseded). Basically I want interactive control over all parts of the phone via a simple serial (or similar) connection. It seems to me that all of this is possible - we need an app developed that can send and receive simple serial commands and data (via bluetooth or USB OTG) and access the phone's resources and act on the commands sent by the microcontroller.

I read the post on using App Inventor to control an Arduino (http://hackaday.com/2013/02/15/beginners-androidarduino-example-shows-the-power-of-app-inventor/) and was amazed at how accessible this was. I don't know how to program Android apps but I might be able to do this.

So I'd like to get some support here please - do you think this is a good idea? Can you help with development of this? Can you program android apps or components of them or gather up open source software components? How do I even start to develop a community-driven open source project like this (I've never done this before)?

Let's get this off the ground!
Benjamin Franzmayr
Palmerston North, New Zealand

[MS3FGX]

Why wouldn't you just write software for the smartphone then?

What is the point of limiting yourself to running code on the MCU and interfacing with the phone when you have the ability to run code natively on the phone to begin with?

[bkf11]

Hi MS3FGX

Because:
- I don't have the ability to write code for smartphones. I don't have time or desire to learn this - I already know how to program my microcontroller. With a serial connection, anything from Arduino to Raspberry Pis to computers to (in my case) Picaxe or any other microcontroller that people have learnt to program with, can all use the same app/resources.
- microcontrollers are designed to control and interface with other electronics - probably more so and better than a phone. Advantages include low power consumption (they could switch the phone on and off as required), lots of inputs/outputs, simple commands for interfacing, easy to learn programming language etc.

Cheers,
Benjamin

[UAirLtd]

I don't know if this is what you were intending, but how I understood your response to MS3FGX was this: "I don't have the ability to write code for smartphones, I don't have the desire to learn it. I just have this cool app idea and I would like someone else to write it for free."

This response certainly came across as having a misplaced sense of entitlement, whether you are like that or not, I will not judge. What I will say is this: it is going to be challenging for you because of the nature of this app: the very same people who CAN write an app, are paradoxically the same people who could live without the app. The skills that they need to write an app are the very same skills the app is designed to allow a user to accomplish things without.

That's not to say that they wouldn't need the app, it would still be useful for rapidly prototyping, but I would certainly rather just write whatever damn app I needed at the time to get the sensor data instead of slaving away at the significantly more difficult task of producing a configurable app that was flexible enough yet simple enough for everyone to use.

And therefore the only reason for someone to build an app like this for free is because they are genuinely altruistic about providing others with some great tools, and don't mind putting in significant effort to allow others to do that.

Your challenge is therefore to find such people, or pitch your idea to a group who would financially benefit (however indirectly) from producing such a tool. Development boards makers are the obvious choice: Arduino, mbed, etc. but there is also people who seek to expand their portfolio of works, or those who need to prototype stuff a lot, etc.

[bkf11]

Hi

Oh no, I really don't want to come across as entitled or selfish - just trying to say that not everyone can "just" write their own software . I see this as a useful resource for many people so I'm trying to find others who would find it useful. I find that if I expand my point of view, projects that would not make sense to do just for myself, make a lot of sense if they can be used by many other people. This is one of those situations. If I just want one GSM module I'll buy one, but with some effort we could create much more value. Isn't that what the Open Source movement is about?

"...the only reason for someone to build an app like this for free is because they are genuinely altruistic about providing others with some great tools, and don't mind putting in significant effort to allow others to do that.
Your challenge is therefore to find such people, or pitch your idea to a group who would financially benefit (however indirectly) from producing such a tool."

Well, yes, that is who I'm trying to find. There are people out there who do that and there would be self-benefit - they don't have to be 100% pure and altruistic . I was thinking more of hobbyists who can write code. Any suggestions of people or where else to look are welcome.

In fact I have a use for this at work (for remote monitoring of a scientific instrument) and can provide some funding to pay for development work. I'm looking around at the moment for someone who could be paid to make a start and write a core app that can be expanded on in the future.

Thanks for your input UAirLtd.

Benjamin

[UAirLtd]

Yeah, that's all understandable, I guess I misread the response without correctly taking into account the context. I also wanted to highlight the paradox because I encountered exactly the same on a similar project - I though to myself: "I wish there was an easier way to do this without having to learn all this stuff", which was followed by the thought "maybe I should make something to help", and that was followed by the realisation that by the time I'd gone and built something to make the task easier, I'd had accumulated all the necessary skills to no longer need that tool.

[bkf11]

Yes, I can see how that happens. I have a full time job, 4 children and other things to keep me busy so while I'd love to learn how to code, I realistically don't have much time. So am trying to see if I can coordinate something instead.

Perhaps my silly avatar doesn't help in how people perceive my responses... I'm really a very nice person!

Benjamin

PS nice quadcopter!