Build a REST API For Your IoT Data Using MongoDB, Node-RED, and Bluemix

In my previous post I talked about how to use the IBM IoT Cloud and Bluemix to build apps based on data coming from an TI Sensor Tag.  One of the apps I showed was Node-RED, however it was very basic in that it only printed the data to a debug node.  In this video I show you how you can store that data in a MongoDB and create a REST API for the data all in Node-RED without writing any code!  Enjoy!

Bluemix ToDo App Updates

The ToDo apps for Bluemix was one of the first sample projects I published after becoming a developer advocate for Bluemix.  Even though the apps are simple, I still think they are some of the most meaningful samples we have.  Today I have made them even better ;)  Since Bluemix went GA we introduced the MongoLab service and an enhanced Cloudant service.  I have decided to change the ToDo apps so they take advantage of both of these new services.

Why change something that is already good?  The old community Mongo service was unreliable, it sometime lost connections with the app, and it was so popular that it was often running out of space to create new DBs.  The old Cloudant service was essentially a user-provided service.  This meant you had to go out to Cloudant signup for an account, create a database, and get an API Key and Token before you could even deploy a ToDo app that used Cloudant.  This was a huge pain.

With the new MongoLab service you get a reliable, production quality Mongo DB :)  Plus you also have the benefit of using the MongoLab console to see the contents of your DBs.

The new Cloudant service now automatically generates an API Key and Token for you when you create the service.  No more having to leave Bluemix to signup for Cloudant and all the ugly stuff :)

I have updated the Bluemix ToDo apps (all languages) to use both of these new services.  Hopefully this makes something that was already good even better, enjoy!

Bluemix and the Internet of Things

Want to take a guess as to how many devices will be connected to the internet by the end of 2015?  Give up?  15 billion.  Yup ,that’s right, that is 15 followed by 9 zeros.  By 2020 that number will increase to 40 billion!

internet-of-things-connected-devices

All of these devices make up what is called the internet of things.  As you might imagine with the number of devices growing there have been a number of new technologies emerging that help developers connect and use the data comming from these devices.  One of the more important ones is MQTT.  “MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport.”  In response to the growing number of devices and the clear importance of gathering information from these devices IBM has developed an Internet of Things (IoT) Cloud which at the core has an MQTT instance that developers can use.  This offering is currently in beta but is quite functional and can be used to publish and retrieve data from your connected devices. Of course once you have a device connected to the IBM IoT Cloud you want to build an app around the data coming from that device.  This is where Bluemix comes in.  Using a combination of the IBM IoT Cloud and IBM Bluemix you can have a complete cloud based solution for your IoT apps.  Bluemix already has a service that is part of its catalog to connect to the IBM IoT Cloud.  In addition to the service, Bluemix has a boilerplate for the internet of things which stands up a Node-RED instance allowing you to design flows for your IoT data.

Screen Shot 2014-07-15 at 9.18.37 AM Screen Shot 2014-07-15 at 9.18.54 AM

As part of IBM’s IoT Cloud there are a number of “recipes“.  Recipes are basically example IoT scenarios that can get you up and running quickly.  The recipes contain instructions on how to setup the device as well as provide you with the code needed to connect the device to the IoT Cloud and publish data.  What makes using these recipes even easier is the Quickstart service that is part of the IoT Cloud.  When using the IoT Quickstart you don’t have to sign up for anything or register any devices, you just run the code given to you in the recipe, head to the qucikstart page, enter the MAC address of the device, and you can see the data being published to the IoT Cloud.  There are some limitations to using the Quickstart, but to get a device up and running quickly it couldn’t be easier. Being new to the whole IoT concept I decided to pick one of the recipes to experiment with.  I ended up choosing the Beaglebone Black and TI Sensor Tag recipe.  The main reason I chose this one was because the Sensor Tag had a number of sensors on it that you could use for different and interesting use cases.  Plus the Beaglebone Black, Sensor Tag, and Bluetooth LE USB adapter costs less than $100.  After my Beaglebone Black and Sensor Tag came in I was able to quickly follow the instructions in the recipe to start publishing data to the IoT Cloud.

Of course like any good developer I started to look at the code provided in the recipe.  I noticed that what it was doing was gathering data from all the sensors on the Sensor Tag and then publishing it under a single topic to the IoT Cloud.  I thought it would be better to publish multiple topics so consumers can subscribe to just the data they need.  I also wanted to use the left and right buttons on the Sensor Tag and I thought you would see a lag between the time a button was pressed and the time the data was published to the IoT Cloud if you had to wait to gather data for all the sensors first.  I quickly realized that if you wanted to publish more than one topic to the IoT Cloud you could not use the Quickstart.  You actually had to sign up for an account and register the device.  This ended up being really easy as well, given there was a recipe describing how to connect a registered device to the IoT Cloud.   After some time coding I was left with 1 project with three components to it.

  • The first component publishes data from the Sensor Tag to the IBM IoT Cloud and is written in Node.js.  It publishes multiple topics for the different types of data coming from the Sensor Tag and requires you to register the device publishing the data.
  • The second component consumes the data from the Sensor Tag that was published to the IoT Cloud.  It contains 2 web apps, one which visualizes the values of the sensors and another which allows you to control an HTML5 slide deck with a Sensor Tag.  The server portion of the webapp is written in Node.js.  The Node.js code connects to the IoT Cloud and subscribes to the topics from the first component (the publish component).  When the Node code receives data from the IoT Cloud it published that data via web sockets to a web application.  Since this component is just a Node.js webapp it can easily be deployed to Bluemix.
  • The third component also consumes the data from the Sensor Tag but it consumes it via a Node-RED flow.  Again you can run this Node-RED flow in Bluemix.

The code for this project can be found on GitHub.  The READMEs contain many more technical details.  Below are two videos demoing the code.  The first one is a short video demoing one single use case.  The second video goes more in depth and demos using the Beaglebone Black.

 

Bluemix Virtual Meetup – Mobile Cloud Services

We have scheduled our next Bluemix virtual meetup for July 23rd at 10 AM EST.  We will again be using Google Hangouts (hopefully without the audio problems this time).  For this meetup we will be focusing on the mobile cloud services in Bluemix, there will be plenty of information and demos to help you get started.  At the last virtual meetup we had a lot of questions on mobile so I figured this would be a good topic.  You can download this .ics file to add the event to your calendar and be sure to respond whether you will be attending on the Google+ events page.

Screen Shot 2014-06-25 at 10.06.00 PM

Using Custom Domain Names In Bluemix

One of the more common questions I get about using Bluemix is “How do I use my own domain names for my apps in Bluemix?”.  Bluemix allows you to bind multiple routes to any given application so in addition to the mybluemix.net route your application has you can bind any number of other routes as well whether they use the mybluemix.net domain name or your own domain name.  Here is a short video of how you can use your own domain names in your application routes.

Using The Mobile Data Service In Bluemix

I have stated to look at the mobile services that are within Bluemix and experiment with how developers can use them in their mobile apps.  I decided to start with the Mobile Data service.  This service provides SDKs for iOS and Android (as well as ones for Node.js and client-side Javascript) which allow you to perform basic CRUD operations from your mobile app.  However before I got started playing with the Mobile Data service I needed a mobile app to experiment with.  Since I am an Apple user and I have iOS devices I decided to stick with iOS.  I found a really good tutorial from Apple which takes the novice iOS developer through building a ToDo app.  After I finished following the tutorial I was left with a working ToDo app, but the ToDos were not persisted so if I close the application all my ToDos were lost.  This seemed like a perfect app to add the mobile data service to in order to persist the ToDos the user creates.  The steps below assume you have finished following the tutorial from Apple and have a working ToDo app.

The first thing you need to do to get started with the mobile data service is go to the Bluemix catalog and create a new application from the Mobile Cloud boilerplate.

Screen Shot 2014-06-13 at 9.54.19 AM

After giving your application a name, Bluemix will provision a new application with a number of services, one of them being the Mobile Data service.  (We won’t use the others in this article, I just use the boilerplate to make it easy to get started.)  After the application is created select the new application from the dashboard and you will be brought to a somewhat familiar application UI.  However the application UI for mobile cloud applications is a little different, notice the application ID.  As we will see later on, this application ID will be important when using the SDK.  In addition there is also a link right next to the application ID to download the SDKs.  Perfect!!

Screen Shot 2014-06-13 at 9.58.13 AM

Click the Download SDKs link to download the iOS SDK and extract it from the zip file.  Next you need to add the SDK to the frameworks for your ToDo project in XCode.  To do this drag and drop the IBMBaaS.framework and IBMData.framework files into the frameworks folder in XCode.

Screen Shot 2014-06-13 at 10.39.54 AM

Once you drag and drop the files you will see a dialog containing options for adding the files, these are the options I chose.

Screen Shot 2014-06-17 at 2.25.58 PM

Next our native application will need to know what our mobile cloud application ID is in order to use the SDK.  It is best to externalize this into a separate file, a .plist file to be exact.  Right click on the supporting files folder in the project navigator in XCode and select New File from the context menu.  In the iOS section of the New File dialog choose Resource and then choose Property List.  Give the file the name configuration.

Screen Shot 2014-06-17 at 2.32.08 PM

Screen Shot 2014-06-17 at 2.32.22 PM

Select the new configuration.plist file in your supporting files directory and add a new property called applicationId.  Its value should be a String and the value should be the application ID from the mobile cloud application you created in Bluemix.

Screen Shot 2014-06-17 at 2.34.05 PM

Now we need to read in the new plist file in our code, extract the application ID from it, and initialize the iOS Mobile SDK with it.  This code can live in the AppDelegate.m file within the didFinishLaunchingWithOptions:launchOptions method.  First we need to add some imports to the file in order to get things to compile.  Add the following imports after the existing imports.

Here is the implementation of didFinishLaunchingWithOptions:launchOptions which reads the plist file and initializes the SDK.

Now we need to make some changes to our model object, ToDoItem, to make it so the IBM Mobile Data SDK can transform it into JSON and persist it to the DB.   Open ToDoItem.h.  The ToDoItem model currently extends NSObject, but we need to change it so it extends IBMDataObject from the SDK.  In addition, we need to add the modifiers nonatomic and copy to the properties of the interface.  Here is what my ToDoItem.h file looked like after doing this.

You will notice that I took out the creationDate property from the Apple tutorial.  The tutorial said to add this but by the end  of the tutorial I did not see it being used anywhere so I removed it.  I also changed the type of completed to NSNumber.  The IBM Mobile Data SDK seemed to have a problem converting non-pointers to JSON (probbaly something here I am not understanding) so I changed the type to NSNumber and made it a pointer.

We also need to make a few changes in the ToDoItem implementation file, open ToDoItem.m.  We need to add @dynamic to completed and itemName to provide getters and setters for the Mobile Data SDK to use.  We also need to implement the method dataClassName which provides the object name that the objects will be persisted under in the mobile backend.  In addition we need to override the initialize method so we can call the registerSpecialization method from the super class.  This creates a mapping between the Objective-c class and the JSON object.  Here is what my ToDoItem.m file looks like after doing this.

Most of the work to manipulate the data in the Mobile Data service on Bluemix happens in the ToDoListTableViewController.m file.  First lets tackle storing new ToDos to the backend.  When a new ToDo is added in the UI the unwindToList:seague method is called, so this is where we want to persist our ToDoItem object to the Mobile Data backend.  This can be done by calling the saveInBackgroundWithCallback method in our ToDoItem.  This method is coming from the IBMDataObject class which our ToDoItem extends.  Here is what my unwindToList method looks like.

After we create new items and persist them to the Mobile Data service we need to retrieve them when the app loads and display them in the list.  We can do this in the loadInitialData method by using the query class-level method on the ToDoItem class.  Here is what my loadInitialData method looks like.

Next we want to address the update case, so when you make a ToDo as done that change is updated in the Mobile Data backend.  This happens in the didSelectRowAtIndexPath:indexPath method.  When an IBMDataObject is updated we can use the saveInBackgroundWithCallback method to save the updated object to the DB.  With some small modifications to the existing method code we can save updates.  Notice in the code below that we also make the necessary changes to deal with the new type of the completed property on the ToDoItem.

Because the completed property of the ToDoItem is no longer a boolean we also need to make a small change in the cellForRowAtIndexPath:indexPath method which adds and removes the check box in the UI.  Right now the if statement which checks the value of the completed property will always return true no matter what the number value is.  To fix this we just need to translate the NSNumber to a boolean.  Here is the updated method.

That is it as far as coding is concerned so lets make sure our code compiles.  If you do a build by going to Product -> Build you will see the build fail with a bunch of build errors.  This is because we need to add some linker flags.  To do this, click on the ToDo List project in the project navigator and then select the ToDoList target.  Next click the Build Settings tab and make sure All is selected.  Under the Linking section select Other Linker Flags.  You will need to add the -ObjC and -lsqlite3 linker flags.  Here is a screenshot which describes this a little better.

Screen Shot 2014-06-17 at 5.11.26 PM Now go to Product -> Build and your project should now build successfully.

Now that our code compiles we can try running the application in the iOS emulator, click the Play button in the upper left hand corner of XCode to launch the emulator.  As you add  and update new ToDos they should be persisted to the mobile data backend.  You can verify this by going to your application in Bluemix and clicking the Mobile Data service on the left.

Screen Shot 2014-06-17 at 9.37.30 PM

In the Manage Data section of the console you can browse the data stored in the DB from your application.

Screen Shot 2014-06-17 at 9.39.51 PM

I hope this gives you a good idea on how you can use the Mobile Data service in Bluemix to persist data for your native mobile applications.  There are many more features of the service that I did not mention in this post, but you can learn about them by reading through the documentation for the service.

Access Service Info With Spring Cloud

Adding services to your application to integrate additional functionality is one of the primary benefits of using a PaaS like Bluemix.  Whether that is a database, a mobile push service, or some type of analytics service, you just pick what you need and bind them to your application.  You don’t have to install software of setup VMs, just point and click.  After you have done that you are always faced with the same problem, you now need to access the service info in the VCAP_SERVICES environment variable.  This info usually consists of some type of credentials or API keys, something that authorizes you to use the service in one way or another. As everyone probably knows, the VCAP_SERVICES environment variable is a JSON object, so extracting information from it just involves parsing JSON.  In Java you have many choices for doing that which I will not go into here, but it is a fairly simple task.  If you are going to be using more than one service in your application your probably going to want to write some type of utility class which allows you to extract arbitrary info for different services by passing some type of unique identifier for the service (typically the service name).  This will return you more JSON to work with.  Again, none of this is hard stuff, even an entry level developer.  However it is kind of monotonous, and when you think about the fact you need to implement this same pattern for every application you deploy to Bluemix, it is unbearable. A helper library would make our lives as Java developers better.  Apparently I am not the only person to think this because one exists :)  It is called Spring Cloud.  Don’t let the name fool you, this library is not limited to just Spring apps, it can be used by any Java app, but if you are using Spring there are some added benefits.

Here is a great article describing the benefits of using Spring Cloud, it also contains a sample app with code to get you started.  The article references using the library with Cloud Foundry and Heroku, but since Bluemix is built upon Cloud Foundry it all works with Bluemix as well.  If it helps you when reading the article just replace the words “Cloud Foundry” with Bluemix :)  The GitHub repo for the project can be found here.  If you are a Java developer, I highly suggest you take advantage of this library.  I have use the library in both the Java ToDo app and the Session Questions app.