June 12, 2019
by Atin Mathur
The idea behind the platform.ai mobile app is to help users test their deep learning models in real-time using their smartphones. After creating a new project on platform.ai, users can use their smartphone camera or gallery to select a picture and classify it using their own deep learning models on platform.ai.
Fig 1. How to access QR code in Platform UI. Step one (above), Step two (below)
Fig 2. Scan the QR code from the expo client app and start testing your application on your Android or iOS smartphone.
$ cd expo-prediction
$ npm install # Install all the required packages in package.json in the repo.
$ expo start # launches the app on localhost
Now, you can open the expo-prediction repo in any code editor and modify the code and the changes would immediately be reflected on your simulator or smartphone.
Fig 3. A) Home Screen: Scan the Project QR code on platform.ai UI, B) Test Model Screen: Select an image from Camera or Gallery. C) Test Model Screen: Prediction APIs returns back the class labels.
We have used the BarCodeScanner component out-of-the-box from the expo sdk. Further, we have used AppLoading to preload and cache assets beforehand for better user experience. On running for the first time, the app asks for user permissions to access the camera and gallery. Only, if the permission is provided, will it show the QRcode scanner screen. Once the user scans the QR code which has the project configuration, there is a test for its correctness and if it passes the test, we navigate (using react-navigation) to the TestModel screen and pass on the project configuration using the navigation state variable.
We use ImagePicker component from the expo sdk to choose pictures from Camera or Gallery and have provided 2 separate buttons for each case. Using ImagePicker API we extract the base64 format of the selected image. Before sending a request to platform.ai servers, we create a JSON object and store relevant “values” into “keys”.
The access token is provided in the project configuration (QR code) on the home screen. On selecting the image, it automatically starts the process of classifying the image.
We have tried to handle all the possible errors that could occur during the classification process. In our testing, we have these most common errors that could occur:
404: Server not found
413: Image Size too Large
500: Internal Server Error
502: Bad Gateway
503: Service Unavailable
In the case of successful classification from the server, it sends a 200 status code. Otherwise, any of the above error codes are sent. We have created a dictionary of all the HTTP errors to display the relevant message accordingly to the user.
Platform AI Mobile App Demo
We are soon going to release the platform.ai mobile app on the respective app stores.