Microsoft teamed up with Gump Come, a provider of smart vending machines, to develop a way to monitor the working status of its machines and to get real-time information on replenishment needs.

Customer

Gump Come is a leading AI innovation-oriented provider of smart vending machines and smart micro-supermarkets in China. These machines integrate the use of cameras, multi-touch, voice interaction, and big data analysis to realize new human-computer interaction patterns.

Pain point

To date, Gump Come has no solution that can monitor vending machine health status or get real-time information on whether a device is working. Any device needing attention usually requires two onsite visits—once to diagnose the issue and again to return with the component to fix the issue. All this adds to the cost of maintaining the machines.

The Gump Come goods manager must check on the selling status of the vending machines to see if they need replenishing. They want to develop an app for this. The goods manager will need training to learn to use the app. They want to get this information only through text or voice on the web app, and to reduce their cost of installing, learning, and updating the application.

Solution

  1. Use Azure IoT Hub to collect data on supplies and device health for all vending machines and send to the uniform management platform for daily monitoring. This will help the staff to diagnose device issues online, which can save on maintenance costs and help realize predictive maintenance cost savings.

  2. Based on Bot Framework and LUIS Services, build a vending machine management bot for an operator to check on vending machines and the status of their goods supply.

In this Smart Vending Machine solution, Gump Come will use the following Microsoft technologies:

  • Azure IoT Hub
  • Stream Analytics
  • Azure SQL Database
  • Power BI
  • Bot Framework

Architecture

The Gump Come Smart Vending Machine solution architecture can be represented as follows:

  • The Gump Come vending machine client will send its device status data to the Azure IoT Hub every 3 minutes. This data includes device status, goods channel status, and temperature.

  • Stream Analytics will transfer this data into three tables on Azure SQL Database, then Power BI will synch these table updates into the admin dashboard.

  • The Gump Come bot will offer voice and text query features to the operator to check the vending machine device status and goods channel status.

Figure 1. Gump Come Smart Vending Machine solution architecture

Gump Come Smart Vending Machine solution architecture

Note: Power BI is not generally available in China; Stream Analytics can’t configure Power BI as an output stream in this phase.

Device used and code artifacts

Microsoft’s China DX Technical Evangelist team and the Gump Come dev team split the engagement into four segments:

  • Vending machine IoT SDK integration
  • Vending machine Azure IoT Hub, Stream Analytics, and Azure SQL Database deployment
  • Data visualization in Power BI
  • Vending machine administrator bot dev

Vending machine IoT SDK integration

In this segment, the Gump Come team focused on vending machine client Azure IoT Hub SDK integration. The Gump Come vending machine client is the Android OS. The Microsoft DX team helped the Gump Come client dev team to code the Azure IoT Hub Android SDK in their client, set up and register the vending machine client with the Azure IoT Hub, connect with the Azure IoT Hub, and send vending machine device status data to the Azure IoT Hub. The Android client code screenshot is as follows:

package com.gumpcome.iot;
import android.os.SystemClock;

import com.gumpcome.kernel.debug.GumpLog;
import com.microsoft.azure.iothub.IotHubEventCallback;
import com.microsoft.azure.iothub.IotHubStatusCode;
import com.microsoft.azure.iothub.DeviceClient;
import com.microsoft.azure.iothub.IotHubClientProtocol;
import com.microsoft.azure.iothub.IotHubEventCallback;
import com.microsoft.azure.iothub.IotHubMessageResult;
import com.microsoft.azure.iothub.IotHubStatusCode;
import com.microsoft.azure.iothub.Message;
import com.microsoft.azure.iothub.MessageCallback;

import java.io.IOException;
import java.net.URISyntaxException;

/**
 * @author LiuPeng @GumpCome
 *         LiuShijun @ Microsoft
 *         Created at 2016/8/30.
 */
public class IotHelper {
    private static final String TAG = "IotHelper";
    String connString = "HostName=iotxxx.azure-devices.cn;DeviceId=myFirstJavaDevice;SharedAccessKey=od1/RiE0ItHtbuewQ/xxxxxxxx";

    public void SendMessage(String message) throws URISyntaxException, IOException {
        IotHubClientProtocol protocol = IotHubClientProtocol.AMQPS;
        DeviceClient client = new DeviceClient(connString, protocol);

        try {
            client.open();
        } catch (IOException e) {
            GumpLog.e(TAG, "Iot连接异常:" + e.toString());
        }
        try {
            Message msg = new Message(message);
            msg.setProperty("messageCount", Integer.toString(1));
            System.out.println(message);
            EventCallback eventCallback = new EventCallback();
            client.sendEventAsync(msg, eventCallback, 1);
        } catch (Exception e) {
        }
        SystemClock.sleep(2000);
        client.close();
    }

    protected static class EventCallback implements IotHubEventCallback {
        public void execute(IotHubStatusCode status, Object context) {
            Integer i = (Integer) context;
            GumpLog.d(TAG, "IoTHub 返回的消息 " + i.toString()
                    + " ,状态:" + status.name());
        }
    }

    public void receiveMessage() throws URISyntaxException, IOException {

        IotHubClientProtocol protocol = IotHubClientProtocol.AMQPS;

        DeviceClient client = new DeviceClient(connString, protocol);
        {
            MessageCallback callback = new MessageCallback();
            Counter counter = new Counter(0);
            client.setMessageCallback(callback, counter);
        }
        try {
            client.open();
        } catch (IOException e) {
            GumpLog.e(TAG, "Iot连接异常:" + e.toString());
        }
        SystemClock.sleep(2000);
        client.close();
    }

    protected static class MessageCallback implements com.microsoft.azure.iothub.MessageCallback {
        public IotHubMessageResult execute(Message msg, Object context) {
            Counter counter = (Counter) context;

            GumpLog.d(TAG,
                    "Iot收到的消息:" + counter.toString()
                            + ",内容: " + new String(msg.getBytes(), Message.DEFAULT_IOTHUB_MESSAGE_CHARSET));

            int switchVal = counter.get() % 3;
            IotHubMessageResult res;
            switch (switchVal) {
                case 0:
                    res = IotHubMessageResult.COMPLETE;
                    break;
                case 1:
                    res = IotHubMessageResult.ABANDON;
                    break;
                case 2:
                    res = IotHubMessageResult.REJECT;
                    break;
                default:
                    throw new IllegalStateException("Invalid message result specified.");
            }

            GumpLog.d(TAG, "Responding to message " + counter.toString() + " with " + res.name());

            counter.increment();

            return res;
        }
    }

    /**
     * 作为消息的回调
     */
    protected static class Counter {
        protected int num;

        public Counter(int num) {
            this.num = num;
        }

        public int get() {
            return this.num;
        }

        public void increment() {
            this.num++;
        }

        @Override
        public String toString() {
            return Integer.toString(this.num);
        }
    }
}

Vending machine Azure IoT Hub, Stream Analytics, and Azure SQL Database deployment

In this segment, the Gump Come team deployed Azure IoT Hub, Stream Analytics, and Azure SQL Database on Azure. First, as a Gump Come proof of concept (POC) for Azure IoT Services, Gump Come created one free IoT Hub in their subscription, deployed Stream Analytics, and created their device status database on Azure SQL Database. Next, Gump Come technical decision makers, based on this POC result, plan to release this IoT solution to their 2,000+ production devices in China.

Figure 2. Azure IoT Hub configuration in Azure

Azure IoT Hub Configuration in Azure

Figure 3. SA input streaming configuration in Azure

SA input streaming configuration in Azure

Figure 4. SA output streaming configuration in Azure

SA output streaming configuration in Azure

Data visualization in Power BI

The Gump Come team presented their device status data and goods channel data through Power BI. They liked Power BI’s excellent graphics and ease of use for the business operator. But because Power BI isn’t generally available in China, we couldn’t connect Stream Analytics directly with Power BI. Instead, the Gump Come team stored the data from Stream Analytics to Azure SQL Database, and used Power BI to connect to Azure SQL Database.

Figure 5. Gump Come vending machine status data in Power BI

Gump Come vending machine status data in Power BI

Vending machine administrator bot dev

Here, the Gump Come team developed the vending machine management bot, which allows the vending machine operator to check the vending machine device status and goods status through a mobile phone.

Figure 6. Vending machine management bot

Vending machine management bot

Opportunities going forward

As a result of this technical engagement, the Gump Come technical and business decision makers plan to roll out this IoT vending machine solution to all 2,000+ devices in China by the end of 2016, hosted entirely on Microsoft Azure.

The Gump Come team will integrate the vending machine management bot app with Wechat in the coming months.

Great team

Figure 7. The Gump Come hackfest team

Gump Come hackfest team

Figure 8. The Gump Come smart vending machine deployment in Microsoft Technical Center

Gump Come smart vending machine deployment in Microsoft Technical Center

Special thanks to the Gump Come team, the Microsoft China DX Technical Evangelist team, and the Microsoft Audience Evangelism team.

This project team included the following participants:

  • Wenbin Cao – Gump Come CTO
  • Hongjie Sun – Gump Come CMO
  • YunLong Dong – Gump Come Chief Architect and IT Pro Manager
  • Peng Liu – Gump Come Software Engineer
  • Yan Zhang – Microsoft DX Audience Evangelism Manager
  • Shijun Liu – Microsoft DX Technical Evangelist
  • Leon Liang – Microsoft DX Technical Evangelist
  • Yuheng Ding – Microsoft DX Technical Evangelist
  • Zepeng She – Microsoft DX Technical Evangelist
  • Lit Li – Microsoft DX Technical Evangelist