Bluemix UI Updates: Bluemix Pumpkin

Trick or treat! Halloween is here in the U.S., and we have a fresh round of Bluemix announcements and Bluemix UI updates. Keep reading for details on all of the new “treats” this month:

  • New IBM and Twitter partnership
  • Addition of Ustream Service<
  • Introduction of IBM DataWorks and IBM dashDB Services
  • New Object Storage Service<
  • General Availability of Internet of Things Foundation
  • Updated Buildpacks for Java and Node.js
  • Enablement of High Availability for Data Cache and Session Cache Services
  • Enhanced UI functionality, such as:
    • Improved Catalog Filtering
    • Enhanced Pricing Overview
    • Removal of “Add-on” Terminology
    • Miscellaneous Usability improvements and Defects

Bluemix UI Updates: Catalog Filtering

Updated: Nov. 22, 2014 to reflect deprecation of env.log.

I’m a fan of the cfenv package for Node.js written by Patrick Mueller. It parses environment info in a Bluemix (or Cloud Foundry) application, and provides functions to make it easy to retrieve all of the service data you need from VCAP_SERVICES. It also gives access to other important attributes for port, host name/ip address, URL of the application, etc. On top of that, it detects whether your app is running locally or in the cloud. And, when running locally, it provides handy defaults.

I’ve written a simple wrapper (a local module called cfenv-wrapper) to make local development of Bluemix/Cloud Foundry apps a little easier. My code parses a local copy of your app’s environment data, and extracts the data for VCAP_SERVICES and VCAP_APPLICATION. Then, it passes that information to cfenv’s initialization function, getAppEnv. After this initialization, you can use the same cfenv interface just as if you were running in the cloud.

The local environment data can be in JSON or properties files meeting the following requirements:

  • env.json – JSON file with VCAP_SERVICES info as retrieved by cf env. If env.json is present, the code will also load env_custom.json, which should hold JSON representing your app’s user-defined environment variables.
  • env.log file (deprecated) – If there’s no env.json file, the wrapper will load a local copy of your app’s env.log file. This is provided for backwards compatibility with earlier versions of cfenv-wrapper. After CF 182, CF stopped providing an env.log file for security reasons.

NOTE: Many cloud services can be accessed with no changes when running locally. In Bluemix, a partial list of these includes Cloudant, Pitney Bowes, and Twilio. In those cases, you can use env.json and env.log with the exact data provided by CF. However, there are services that don’t yet allow connections from outside of Bluemix. For those services, you would need to modify your local file so that it uses info specific to installations of those services in your local environment.

New Functions

My wrapper also adds two new functions, not in the original cfenv interface:

  • getEnvVars – returns JSON data structure containing all environment variables. When running locally, it returns the data from the env_custom.json or env.log files. When running in the cloud, it will return the standard process.env runtime variable.
  • getEnvVar(name) – returns value of the environment variable with the given name. When running locally, it will try to pull the value from the env_custom.json or env.log files. If it can’t find it, it falls back to the standard process.env runtime variable.

These serve to give you a consistent interface for environment variable resolution in both environments.

Code on Github

The code for cfenv-wrapper (and a working sample) is available on GitHub:

The repository’s README file contains full details on how to get the code and run either locally or in Bluemix.

Run Live Code

You can run a working version of the code deployed to Bluemix by using the link below:

When you run it, you should see something like the following screenshot split into three sections for Base Info, Services, and Environment Variables. The Services section shows the names of any services the app is bound to. The Environment Variables section shows the standard environment variables common to Cloud Foundry apps plus any custom environment variables. NOTE: The code intentionally prunes VCAP_SERVICES from the Environment Variable output because it typically contains sensitive service credentials.

Screenshot of Running cfenv-wrapper

Below, you can see the details for my app in the Bluemix UI. Notice the list of services matches the cfenv-wrapper output:

Bluemix UI: App Details for cfenv-wrapper

Also, for this deployment of the app, I’ve set three custom environment variables (e.g., CUSTOM_ENV_VAR1, CUSTOM_ENV_VAR2, and CUSTOM_ENV_VAR3).

Bluemix UI: Environment Variables for cfenv-wrappe

You can set custom environment variables either in the UI (above) or via the command line: cf set-env cfenv-wrapper CUSTOM_ENV_VAR1 "Value 1"

Getting Environment Info For Local System

JSON Format

To see a JSON representation of your app’s environment data, you can run the following cf command: cf env APP_NAME

Then, copy the JSON (which starts right after the System-Provided header and ends just before the User-Provided header) into a file named env.json. This file should be in the same place on your local file system that you put the code. That is, as a peer to the server.js file.

If you have user-defined environment variables, put them into a file name env_custom.json. For example, if cf env shows the following user-provided environment variables:

User-Provided:
CUSTOM_ENV_VAR1: Value 1
CUSTOM_ENV_VAR2: Value 2
CUSTOM_ENV_VAR3: Value 3

Then, you would want your env_custom.json file to look like this:

{
    "CUSTOM_ENV_VAR1": "Value 1",
    "CUSTOM_ENV_VAR2": "Value 2",
    "CUSTOM_ENV_VAR3": "Value 3"
}
env.log Format

NOTE: This section is only applicable if running a version of CF earlier than 182.

To see the contents of env.log, you can run the following cf command: cf files APP_NAME logs/env.log

Then, copy the output into a file named env.log to the same place on your local file system that you put the code. That is, as a peer to the server.js file.

The Code

server.js

The server.js file is shown below. It initializes the cfenv-wrapper and then uses it to retrieve and display base info, names of services bound to the app, and environment variables.

cfenv-wrapper.js

The cfenv-wrapper.js file is shown below. If running locally, it will try to parse env.json in the working directory. If found and successfully parsed, it will then initialize cfenv. If, not it will try to load and parse env.log. If there’s no success using env.json nor env.log, then the wrapper will just use cfenv with no special initialization. In addition to this initialization, the code provides functions that “wrap” the functions of cfenv.

Bluemix UI Updates: Bluemix & Watson Logos

We’re pleased to announce another round of Bluemix UI updates. This refresh is especially exciting as we welcome IBM® Watson™ services to the Bluemix Catalog. With this addition, you can now rapidly prototype and build cognitive apps in the cloud!

Along with Watson, we’ve made a number of other major improvements since our August release. These include:

  • Upgrade of three services from beta to general availability: Business Rules, Delivery Pipeline, and MQ Light
  • Update of the Cloud Integration add-on, including the ability to create private services
  • Tighter integration of organizations and spaces into the Dashboard
  • Addition of blog and Twitter feeds to the Bluemix landing page
  • Single sign-on with Bluemix Documentation to provide more personalized sample code snippets
  • Numerous usability improvements and bug fixes

Bluemix UI Updates: Watson Comes to Bluemix

MIO Alpha I Strapless Continuous Heart Rate Monitor

Some of my peers in IBM’s Emerging Technology group have been creating some really cool Internet of Things (IoT) demos using IBM Bluemix. Two of them, HR Tracker and Race Tracker, are geared toward fitness and activity tracking. They are both capable of displaying the heart rate and geolocation of exercisers as the values change in real-time. In this post, I’ll describe both sample apps and give an overview of the technology used to build them.

HR Tracker

Last month, I got to be part of an HR Tracker demo at That Conference 2014. For my part, it was a pretty simple process. First, I installed the demo app on my iPhone. Then, I strapped on a brand new, wrist-worn MIO Alpha I Strapless Continuous Heart Rate Monitor, turned it on, and paired it with the phone via Bluetooth. Finally, I hit the Start button in the app. And, before I knew it, a new icon popped up in colleague Mark VanderWiele’s browser showing my heart rate and position. Pretty cool!

HR Tracker Screenshot

And, even cooler, as my heart rate changed on the MIO Alpha, I could see it update almost immediately in the web app.

HR Tracker Screenshot

At this point, I went out for a short walk. While I was gone, Mark could see exactly where I was and how “hard” I was working (via heart rate) in the parking lot and side streets around the Kalahari Resort.

Race Tracker

Race Tracker is similar to HR Tracker, but is geared more toward monitoring data for lots of participants in an event. This morning Bryan Boyd took part in the Medtronic Twin Cities Marathon up here in Minnesota. Of course, running a marathon is way on the other side of the intensity spectrum from my little walk. :)

Via the Race Tracker app (written mostly by Bryan), I was able to “watch” his progress during the race. The animated GIF below (sped up to reduce size) shows Bryan’s data updating at he crossed the 20-mile point (on the St. Paul side of the Mississippi River):

Race Tracker w/ Bryan Boyd in Marathon

I then switched to the satellite view and caught him still going strong at Mile 21:

Race Tracker w/ Bryan Boyd in Marathon (Satellite)

Race Tracker can also send notifications at pre-determined intervals. For example, the tweet below came through the feed as Bryan reached 24 miles:

So, did Bryan finish? I wish I knew! Around Mile 25, data updates stopped being shown in the web UI. I’m assuming his heart kept beating. So, did the battery on his phone or heart rate monitor run out? Once I find out, I’ll try to post an update. :)

Update: I heard from Bryan, and his heart is indeed still beating. He blamed the glitch on “developer error.” But, as a fellow developer, I know those things happen with prototypes. :) Oh, and by the way, Bryan also finished the race in a very respectable 3:57.

Technology

On the day I tried HR Tracker, Mark conducted a session called From Concept to Operation in 30 minutes Using Services and a PaaS. During his talk, he went through the steps needed to build a Bluemix-based backend for these apps. It was a great demonstration of how quickly the “next big idea” can be prototyped and delivered to run on Bluemix.

Architecturally, MQTT ties all of the components together. Every piece either publishes or subscribes to an MQTT Broker (in this case powered by IBM MessageSight). So, for example, when the iPhone app detects a change in heart rate or GPS position, it publishes a message with that information to the MQTT Broker. The MQTT Broker in turn sends the messages to all subscribers.

As you might have guessed, the Bluemix-based backend is a subscriber. Mark started by creating a new app in the Bluemix UI from the Internet of Things Starter boilerplate. The resulting app runs a Node-RED server and is automatically bound to a new instance of the Cloudant NoSQL DB service. With NODE-Red, you can graphically create a flow that subscribes to an MQTT Broker and takes appropriate action. In this case, it acts by writing the information to the Cloudant DB and by sending a tweet if the runner has crossed a mile marker. So, in just a few minutes you can have a Bluemix app processing IoT data from a large number of devices and putting that data right into a database for later retrieval and analysis.

And, in case you’re wondering, the map UI also subscribes to the messages and updates the display as new information arrives.

What’s Next?

Since data is stored as it comes in, all sorts of additional features are possible. The Emerging Tech team already has code in HR Tracker to allow people to “replay” past activities or to compete against themselves (or others) on previous routes.

They’ve also begun exploring interesting things that could be done by applying analytics to all of this data. For example, did someone cheat? Does data indicate someone was grabbing the side of a car for half of their so-called bike ride?

With that, I’ll close with a question to anyone who has made it this far in the post. And, I’d love to see some responses in the comments section. :)

Question: If you had access to all of the data generated by this IoT scenario, what would you be interested to find out?

Bluemix UI: SSL Certificates for Custom Domains

In my previous post Redirecting HTTP to HTTPS with Node.js & Express on IBM Bluemix, I described how you automatically get basic SSL support if you use the default domain when configuring your apps to run on IBM Bluemix. However, if you use a custom domain, you’ll get domain mismatch errors in the browser. These errors are ugly and likely to scare away users. In this post, I’ll show you how to fix this problem by associating your own SSL certificate with your custom domain. This new functionality went live in the August 2014 refresh of the Bluemix UI. Along the way, you’ll learn how to do the following:

  1. Add a custom domain.
  2. Setup an app to use the custom domain.
  3. Configure DNS so traffic is routed to your app.
  4. Upload an SSL certificate for your domain. NOTE: There is a limit of one upload per organization for trial users and four uploads per organization for pay-as-you-go and subscription users (updated Dec. 7, 2014).

Prerequisites

Before we get into the more fun part of the post, let’s cover some important prerequisites that will make things go more smoothly along the way. In particular, you will need to:

  1. Have (or acquire) ownership of a registered Internet domain name
  2. Obtain (or create) an SSL certificate, private key, and (optionally) an intermediate certificate

I’ll say more about these requirements in the next couple of sections.

Registered Domain Name

We will be creating a custom domain in Bluemix, and for it to function properly, you will need to own the domain name that you plan to use. In addition, it must be registered with a DNS hosting service that allows you to make configuration changes. For example, I own the tonyerwin.com domain (which points at the blog you’re reading right now) and use GoDaddy for DNS hosting. (NOTE: GoDaddy is just one of many suitable providers.)

GoDaddy Domain Dashboard

Even if you don’t own a domain or have access to your DNS provider, you can still explore the functionality in this article. But, you will have to edit your hosts file. I’ll say a little more about that option later on in the Configuring DNS for Custom Domains section.

SSL Certificates

What If I Don't Have a Certificate?

So, you want to try this cool feature out, but you're not ready to purchase a certificate from a certificate authority? If so, self-signed certificates could be a good option for you. While not appropriate for production apps, they do have a place during development. Check out my companion post which describes three different ways to create self-signed certificates: Generating Self-Signed SSL Certificates for Use with Bluemix Custom Domains.

When it comes time to associate a certificate with your domain, you will need the following files with the noted attributes:

  • Certificate
    • Digital document that binds a public key to the identity of the certificate owner, thereby enabling the certificate owner to be authenticated.
    • Generally issued and signed by a certificate authority. However, for testing and development purposes you may use a self-signed certificate.
    • File types supported:
      • PEM (.pem, .crt, .cer, and .cert)
      • DER (.der or .cer )
      • PKCS #7 (.p7b, .p7r, .spc)
        • Limitation: Bluemix supports only one certificate per file. That is, you cannot bundle a certificate and intermediate certificate into one PKCS #7 bundle.
  • Private key
    • Algorithmic pattern used to encrypt messages that only the corresponding public key can decrypt. The private key is also used to decrypt messages that were encrypted by the corresponding public key.
    • File types supported:
      • PEM (.pem, .key)
      • PKCS #8 (.p8, .pk8)
  • Intermediate certificate
    • You should use an intermediate certificate to verify the authenticity of the main certificate. Intermediate certificates are typically obtained from a trusted third-party. You might not require an intermediate certificate if using a self-signed certificate for testing your application prior to deploying it to production.

See the Securing Apps article in the Bluemix Documentation for the most current information on supported certificate types.

Adding a Custom Domain

What If I Already Have a Custom Domain?

You may already have a custom domain configured for Bluemix. If so, great! But, please pay attention to the upcoming section on Configuring DNS for Custom Domains as there are some special DNS setup considerations for SSL.

Now that we have the prerequisites out of the way, let’s start by adding a custom domain via the Bluemix UI. (NOTE: You can also create domains with the cf command line tool.)

  1. Log in to the Bluemix UI.
  2. In the banner, open the Organizations menu and choose Manage Organizations.

Bluemix UI: Select Manage Organizations from Banner

  1. In the Manage Organizations pane, make sure your organization is selected in the tree on the left side. Then, click Domains to view the custom domains for your organization.

Bluemix UI: Manage Organizations Top-Level

  1. On the Domains tab, click the Add Domain button.

Bluemix UI: Manage Domains

  1. Enter your custom domain in the text box in the newly added row. As mentioned earlier, this should be a domain that you own and that is registered with a DNS hosting service. For my custom domain, I own the tonyerwin.com domain, and I want to use the bluemix.tonyerwin.com domain for my demo apps that will be deployed to Bluemix.

Bluemix UI: Adding Custom Domain

  1. Click the Save button. After the save operation completes, you should notice a new icon in the SSL Certificate column. We’ll come back to it later, but clicking this icon is how you would start the process of uploading a certificate for your domain.

Bluemix UI: Custom Domain Added

Using a Custom Domain in an App

Now, that we have a custom domain, let’s setup an app that makes use of it. We can either create a brand new app or edit the routes of an existing app. And, in the rest of this section, I’ll walk you through both paths.

Creating a New App With a Custom Domain

If you want to create a brand new app, do the following:

  1. Click on the Catalog in the Bluemix UI banner.
  2. In the Starters section, click on one of the runtimes like Liberty for Java™, SDK for Node.js™, Ruby on Rails, or Ruby Sinatra.
  3. On the details page for the selected runtime:
    • Enter a Name for your application.
    • Optionally change the default Host.
    • From the Domain pulldown, select your custom domain.

Creating Starter App With Custom Domain

  1. Click the Create button to create your app. After creation, you will be taken to the App Details view of your new app.

Editing Routes of an Existing App to Use a Custom Domain

If you prefer to edit the route of the existing app, follow the steps below:

  1. Go to the App Details view for your app. If you created your app in the last section, you’re probably already there. Otherwise, find your app in the Dashboard and click on it.

Bluemix UI: App Details Showing Same App With Gear Menu

  1. Open the “gear” menu in the upper right hand corner of the App Details view, and choose the Edit Routes menu item.
  2. In the Edit Routes dialog, you can add and remove routes. In addition, each route offers a pulldown that includes all available domains and allows you to choose any of your custom domains.

Edit Route Dialog Before SSL Cert Upload

  1. Notice in the screen shot above that my app has one route, ssl-demo.bluemix.tonyerwin.com. Also, note that the SSL status icon to the right of the route shows an open padlock. This indicates we have not yet uploaded an SSL certificate for the route’s domain.

Configuring DNS for Custom Domains

What if I Don't Own a Domain?

You may still wish to experiment with this functionality even if you don't own a domain or have access to your DNS provider. An option for you would be to edit your `hosts` file. The mechanism for doing this will depend on your operating system, but in the end you'll want to add an entry like one of the following for your app's route:
# For an app running in Dallas 
# (ip from running nslookup on
#  secure.us-south.bluemix.net)
75.126.81.68 www.example.com
OR
# For an app running in London
# (ip from running nslookup on
#  on secure.eu-gb.bluemix.net)
5.10.124.142 www.example.com
OR
# For an app running in Sydney
# (ip from running nslookup on
#  secure.au-syd.bluemix.net)
168.1.35.166 www.example.com

At this point, we have an app using a route making use of our custom domain. However, if you tried to access your app via the route, it would fail. To resolve this, you need to do some DNS configuration. Specifically, you need to add a CNAME record that maps your domain to the hostname of the Bluemix router. Since we plan to use our own SSL certificates, the hostname we want to reference is secure.us-south.bluemix.net. NOTE: The host to use varies by the region your app is hosted in:

  • US South: secure.us-south.bluemix.net
  • London: secure.eu-gb.bluemix.net
  • Sydney: secure.au-syd.bluemix.net

The mechanism for adding a CNAME record is going to vary depending on your provider. For GoDaddy, it’s rather simple. I go to the DNS Management configuration page for my tonyerwin.com domain. From there, I click the Add button. This brings up a panel like the one in the screenshot below. On it, I’ve set Type to CNAME, the Host to *.bluemix (the tonyerwin.com part is assumed), and Points to to secure.us-south.bluemix.net.

GoDaddy Add Record Dialog

If you don’t use GoDaddy, you should still find a similar mechanism from your provider. Once you commit these kinds of changes, you will have to wait some period of time for the changes to propagate. Once they do, you should be able to access your app via the browser.

If you’re still unable to access your app after a reasonable amount of time, you might find it helpful to run nslookup. For example, I ran it below for ssl-demo.bluemix.tonyerwin.com, and it properly resolved to secure.us-south.bluemix.net.

$ nslookup ssl-demo.bluemix.tonyerwin.com
Server:  192.168.1.1
Address: 192.168.1.1#53

Non-authoritative answer: ssl-demo.bluemix.tonyerwin.com canonical name = secure.us-south.bluemix.net.
Name: secure.us-south.bluemix.net
Address: 75.126.81.68

Uploading a Certificate for Your Domain

Revisiting the Need for Your Own Certificate

We’re now at a point where you can access your app in the browser using a route that includes your custom domain. But, if you try to access it using the https protocol, you’ll get a browser warning. You can try this for yourself using the link to one of my apps below:

If you follow that link, your browser is going to report that you can’t trust the identity of the server because it identifies itself as *.mybluemix.net rather than the domain in the URL. For example, here’s what I see in Chrome:

Security Warning in Chrome With Custom Domain

So, this finally brings us to the main point of this post — namely, eliminating that server mismatch by using your own certificate.

Time to Upload!

With that, let’s walkthrough the steps to upload your files (certificate, private key, and optional intermediate certificate) that you collected as part of the prerequisites:

  1. Go back to the Manage Domains tab where you first defined your custom domain.
  2. Find your domain in the table and click the Upload Certificate icon in the SSL Certificate column. Recall I pointed out this icon when you first created your custom domain.
  3. For each field on the Upload Certificate dialog, click the Choose button and select the appropriate file.

Bluemix UI: Upload Certificate Dialog (Populated)

  1. Click the Upload button and the upload process should begin. Once the upload is complete, the dialog will disappear and you will be taken back to the Manage Domains table. And, you should also see the icon for your domain has changed to a green padlock.

Bluemix UI: Manage Domains with SSL Uploaded

  1. If you click on the green padlock (which is the View Certificate icon), you will see a dialog much like the one below where you can see all of the important details about the certificate. Note that the dialog also contains buttons that allow you to Replace or Delete your uploaded files.

Bluemix UI: View Certificate

  1. If you also uploaded an intermediate certificate, you can click the root node of the certificate tree at the top of the dialog. This will show you all of the important information about the intermediate certificate.

Bluemix UI: View Certificate with Intermediate Selected

  1. Click the Close button on the View Certificate dialog.\
  2. Optionally, go back to the App Details view for your app and again bring up the Edit Routes dialog. You should see a green padlock next to your route indicating it’s now secured by your uploaded files.

Bluemix UI: Edit Routes Dialog with Route Secured

Verify the Results

Now, if you try to access your route via the https protocol, the browser should declare your site is properly identified and trusted. For example, the screenshot below shows the certificate details for my app from Chrome:

Bluemix UI: Chrome Details Trusted

But, for full disclosure, if you visit my app (see https://ssl-demo.bluemix.tonyerwin.com), you’ll get a slightly different result because I didn’t actually get my certificates from a trusted certificate authority. Instead, I used Keychain Access on my Mac to generate a self-signed intermediate certificate. In addition, I configured the certificate to be trusted on my system. (If you’re interested, I described these steps in my other post on creating self-signed certificates.) Since my intermediate certificate is obviously not trusted on your system, you’ll see a browser warning if you try to access my app. The screen shot below shows the warning I see in Chrome if I try to access my app from another system. While there’s still an error, notice that it’s no longer a server mismatch error. Instead, Chrome tells me the certificate was issued by an entity not trusted by my computer.

Bluemix UI: Chrome Warning Non-Trusted

Redirecting HTTP to HTTPS

So, we’re in pretty shape now, but there’s one more important item to consider. In my previous post on redirecting http to https, I explained that unless you take steps (either in your app code or app configuration), unencrypted http traffic will still be allowed. You can see an example of that if you visit my live app using the http protocol at http://ssl-demo.bluemix.tonyerwin.com. In the other post, I go into the problem in much more detail, and I describe a sample Node.js app that demonstrates one approach to solving it. That sample is also running live using my custom domain (and certificates) at http://redirect-demo.bluemix.tonyerwin.com.

Conclusion

We covered a lot of ground in this tutorial as you learned all about Bluemix support for associating your certificates with custom domains. Along the way, you saw how to add a custom domain using the Bluemix UI, setup an app to use a custom domain in a route, configure DNS so your custom domain resolves to the IP address of the Bluemix router, and finally upload a certificate for a custom domain. I also briefly touched on some related issues (covered in more detail in other posts) such as generating self-signed certificates and redirecting http traffic to https.

Updated August 10, 2015: Remove mention that a certificate must be a wildcard certificate. That limitation is no longer in place.