MI-LAB (MobileInsight Lab)

1. Quick Start

1.1 Create Your Account (optional)

Before you start, you are encouraged to create an account, which will allow you to get access to more data, create an experiemental task of you own, manage experimental tasks and data of your interest, and run more functions provided by MI-LAB. Certainly, you always have an option to log in as guest (with limited permission).

Create your account here. Other options can be found at tutorial.

1.2 Play Three Different Roles

You can play one of three roles or a hybrid one in MI-LAB.

Task owner who publish an experimental task developed by you and call for others (including MI-LAB testbed and volunteering participants) to run the published task and collect data requested by you;
Client contributor who help running experimental tasks via a MI-LAB client on your rooted Android phone or other test equipment;
Data user who browse and use datasets collected and shared by MI-LAB, as well as other services provided at the MI-LAB website. You do not have to participate in data collection to get their access.

1.2.1 Task Owner

As a task owner, you are responsible for developing, releasing and managing your own task app. The task app defines and implements what experiements to run, what data to collect and how to collect. We encourage you to test your app locally and then release it to MI-LAB. You are responsible for privacy issue in data collection (e.g, get an IRB approval if your task app collects any sensitive data to user behaviors). You can manage permissions at the start when you publish your task app to MI-LAB or anytime when you change your mind. Once your task app is released at MI-LAB, you can go to the dashboard to check and monitor task execution and get data collected.

Please refer to this page for more detailed instructions.

1.2.2 Client Contributor

As a client contributor, you are expected to run one or multiple MI-LAB tasks developed by yourself or others. You can run these tasks through MI-LAB clients which currently run on rooted Android smartphones. Data traces will be collected on the run, as programmed by each task app. Collected data will be uploaded to the MI-LAB server upon your consent (which is configurable anytime).

Please refer to this page for more detailed instructions.

1.2.3 Data User

You can choose not to participate any experiments but log in to our MI-LAB website to get access to MI-LAB task data which is released upon the owner’s consent. Other functions, services and information like MI-LAB operation statistics, testbed management, utilitity network analytics functions, are available. Note that access permission to specific task data is controlled by task owners, not MI-LAB.

Please refer to this page for more detailed instructions.

2. Task Owner

MI-LAB is designed to offer task-centric support for experimentation and analysis. Data is collected for certain research task and will be used for this task. MI-LAB task support the following features:

Customized android task app distribution
MobileInsight integration API support
Experimental data management and analysis
Permission management for individual/public to view, execution and data access

To build your own customized task app, please follow the following three steps:

Develop your task
Release your task
Manage your task

2.1 Develop Your Task

You need to create your own task app which can be distributed by MI-LAB and work with both MI-LAB client and MobileInsight.

Before you start, please note that a MI-LAB task should be designed to run along with a MobileInsight plugin.The idea is to let MobileInsight plugin focus on collecting targeted cellular messages and provide run-time analysis, while MI-LAB task app is responsible for the rest needs, e.g. creating or waiting for the desired network environment, generating background data traffic, recording application layer performance metrics, and doing whatever you want as possible, to name a few. That is, given the MI-LAB client and MobileInsight, you only need to do two things:

Create MobileInsight Plugin
Create Android Task App

2.1.1 Create MobileInsight Plugin

MI-LAB releases MobileInsight Plugins for your use. Among them, there are two sample plugins for your attention.

NetLoggerCFG, which collects all cellular signaling messages from NAS layer to PHY layer available at MobileInsight.
NetLoggerMMDiag, which is designed to study mobility configurations and collects signalling messages containing mobility configuration information for multiple RATs including LTE, WCDMA, EVDO, 1xRTT and GSM along with radio signal measurements at physical layer for serving cell and neighbouring cells.

More plugins are released over time. You can use them directly for your task, or create your own MobileInsight plugin. Regarding how to develop MobileInsight plugin, please refer to MobileInsight Develop Tutorials.

2.1.2 Create Your First Android Task App

MI-LAB client supports several interfaces to make the task app co-work with MobileInsight and itself.

You could start with this task demo and use it as a sample template to create your own task app.

Interface with MI-LAB client app

MI-LAB client app uses AIDL to communicate with the task app. To support this feature, you need to make a "net.mobilieinsight.milab" package under the aidl directory and include the following three aidl files. They are all included in the template demo task repo.

    // IMILab.aidl
    package net.mobileinsight.milab;

    // Declare any non-default types here with import statements

    interface IMILab {
        /**
         * Demonstrates some basic types that you can use as parameters
         * and return values in AIDL.
         */

        void pauseMI();
        void resumeMI();
        void sendMsg(String strMsg);
        void insertMsg(String strMsg);
        void basicTypes(int anInt, long aLong, boolean aBoolean, float aFloat,
                double aDouble, String aString);
    }

    // ITask.aidl
    package net.mobileinsight.milab;
    import net.mobileinsight.milab.TaskObject;
    import net.mobileinsight.milab.IMILab;

    // Declare any non-default types here with import statements

    interface ITask {
        int getPid();
        TaskObject getOutput();
        void exit();
        /**
         * Demonstrates some basic types that you can use as parameters
         * and return values in AIDL.
         */
        void basicTypes(int anInt, long aLong, boolean aBoolean, float aFloat,
                double aDouble, String aString);
        void register(IMILab interfaceMILab);
        void run();
    }

    // TaskObject.aidl
    package net.mobileinsight.milab;
    parcelable TaskObject;

MI-LAB client app requires a TaskObject sent from the task app. This should be defined locally within 'net.mobileinsight.milab' package under java directory. This is also included in demo task repo.

    package net.mobileinsight.milab;

    import android.os.Parcel;
    import android.os.Parcelable;

    /**
     * Created by moonsky219 on 5/6/18.
     */

    public class TaskObject implements Parcelable {

        private String TaskName;
        private String TaskDescription;
        private String pathOutputFolder;
        private String pluginNameMI;

        // ......
        }
    }

Your own task app will be launched as an Android service. To make it, you need to declare your main service in the AndroidManifest.xml:

    <service
            android:name=".MainService"
            android:enabled="true"
            android:exported="true"
            android:stopWithTask="false" />

2.1.3 Needed Modifications

Now you come to the most important part. In the MainService, you need to use the interface defined previously and specify the information contained in a TaskObject in the override function public TaskObject getOutput().

The fields consist of the following 4 items:

TaskName: the name of your Task App
TaskDescription: a description of your Task App (it will showed on the MI-LAB Client APP task panel)
PathOutputFolder: the absolute path of the directory which contains all files generated by your task APP and wish to upload to MI-LAB as task logs
MobileInsight Plugin: the name of the MobileInsight Plugin you want to use

    @Override
    public TaskObject getOutput() throws RemoteException {
        Log.i(getString(R.string.tag), "getOutput() is called.");
        TaskObject toSend = new TaskObject();
        toSend.setTaskName(getString(R.string.app_name));
        toSend.setTaskDescription("A demo task using NetLoggerMMDiag");
        String strOutput = new File(Environment.getExternalStorageDirectory(), getString(R.string.app_name)).getAbsolutePath();
        toSend.setPathOutputFolder(strOutput);
        toSend.setPluginNameMI("NetLoggerMMDiag");
        return toSend;
    }

The entrance is the override function run() which will be called by MI-LAB client app once user click "Run" button. You can launch your own activity from there.

    @Override
    public void run() throws RemoteException {
        getApplicationContext().registerReceiver(brPauseMI, new IntentFilter(getString(R.string.tag) + ".MainService.PauseMI"));
        getApplicationContext().registerReceiver(brResumeMI, new IntentFilter(getString(R.string.tag) + ".MainService.ResumeMI"));
        Intent dialogIntent = new Intent(getApplicationContext(), MainActivity.class);
        dialogIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
        startActivity(dialogIntent);
    }

In case you do not need to launch an Activity for any UI, MI-LAB provides an alterative interface: sendMsg(String), which will display task app messages on MI-LAB app console.

    MainService.this.interfaceMILab.sendMsg(getString(R.string.tag) + " started.");

MI-LAB also provides another two interfaces: pauseMI() and resumeMI(), which allow the task app to pause and resume running of MobileInsight plugin.

Note

You don't need to call resumeMI() to start MobileInsight plugin, MI-LAB launch MobileInsight and run the specified plugin once user click the MobileInsight button.

    MainService.this.interfaceMILab.pauseMI();
    MainService.this.interfaceMILab.resumeMI();

2.1.4 What’s More?

Once you finish developing your task app, you need to prepare a task configuration file to describe the task metadata. This will allow you and MI-LAB to manage this task and data to be collected by following MI-LAB policies. At this point, policy management is optional but will be enforced in the near future.

2.2 Release Your Task

Now you can log in to MI-LAB to release your task.

Please log in and go to the dashboard page and click “create task”. You need fill in the required information and upload the required files in order to create and publish your task to MI-LAB. You also need to initilize task permissions (view, executation and data access) at that moment.

Task owner who publish an experimental task developed by you and call for others (including MI-LAB testbed and volunteering participants) to run the published task and collect data requested by you;
Client contributor who help running experimental tasks via a MI-LAB client on your rooted Android phone or other test equipment;
Data user who browse and use datasets collected and shared by MI-LAB, as well as other services provided at the MI-LAB website. You do not have to participate in data collection to get their access.

2.2.1 How to fill the form?

The form contain several information you need to fill.

Name: the name of your task
Description: the description of your task
Plugin: choose it from the existing ones, or create your own plugin by uploading one which was created prior to publishing your task
Taskfile: the apk of the task app to be released. For task app development, please refer to this link
Configfile: the configure file that you upload to contain the information to manage this task and data to be collected. Optional at this moment. )
Applicationid
Task permission:

Viewable: the published task can be seen by other users
Data sharable: the data to be collected will be seen by and accessible to other users
Executable: this published task can be executable by other users
Close Task. The task is by default open (active). Once you close your task, no one can execute your task until you reopen it.

Note

You can change your task permissions anytime when you change your mind. You can manage more permissions provided by MI-LAB. Please refer to this link to learn how to manage your task permissions.

2.3 Manage Your Task

You can manage your own task by updating task files, revising task persmissions, checking execution information and getting data access through the MI-LAB dashboard. You can close the task any time and the closed task will be removed when the participating clients refresh the list of available tasks on client app. You can also control permissions of your task to all the users or one specific user.

2.3.1 View Your Tasks

Log in and click “dashboard”, you will see “My Tasks” and “Other Tasks”. You can click each task to see data collected for this task.

More instructions can be found here

2.3.2 Chanage Permissions

For each task, MI-LAB supports three general permission types to any user (the public) and allows to specify the permission per user.

Viewable: task is visible to any user (the public) only if viewable.
Share: data collected for this task is downloadable to any user only if sharable
Execute: task app is runnable on the MI-LAB client only if executable
By clicking “Manage Task Permission” in the dashbarod, you can change the existing permissions for all your tasks.

By clicking “Manage User Permission” in the above panel or in the dashbarod, you can change the task permissions to specific registered users.

3. Client Contributor

Through MI-LAB client app, you can run tasks, collect data logs and upload them to our server for the task owner to gather all the task data. Note you can choose not to upload data although it is not encouraged in MI-LAB. Buiding an open community counts on you.

3.1 Download and Install

You can download both MI-LAB client app and MobileInsight from this page. You then install them at rooted Android smartphones. We support all the phone models which can run MobileInsight.

3.2 Run a Task

You can run a task with one, two or three steps:

Refresh the list of available tasks and choose one to run. It is not necessary if you have done it already.
After you click the chosen task, you will jump to the task app panel. For the first time, you need to download the task app and install it by following the instructions. Note that the task app may be developed and released by any user and its installation usually asks you to grant certain permissions. Please read it carefully and decide if you want to grant these permissions.
At the task panel, click “run” to start the task app. You can also go to the setting of this task app to change its configurations and stop/start the app anytime. Textbox allows you to insert the tag for the taskround you run.

3.3 Upload logs

After finishing running a task, you could go to the home page of MI-LAB client and press the upload button, which will upload all the collected logs to the server. You could log in to MI-LAB website and check your contribution through the dashboard. By default (see the settings of MI-LAB client), data will be automatically uploaded when the client app goes back to the homeage and WiFi is available. Of course, you can change the upload settings anytime.

3.4 Advanced mode: remote monitor and control

To facilitate your task running and data collection process, you could enable the advanced mode to join our testbed which will allow our testbed management to monitor and control your phones remotely. You can enable this advance mode anytime, that is, you can register your device and join our testbed temporally when you want to use testbed management services to monitor and control experiments, and then deregister your device and leave our testbed when you do not need this service any longer.

When your device is registered to our testbed, a connection will be established with MI-LAB server to monitor the device’s status. Once the remote control is enabled in the client setting, the device can be controlled remotely via the MI-LAB’s testbed management panel. Please refer to this page for more information on device monitoring and control.

For more information on how to run a client app, please refer to this page.

4. Data User

You can log in MI-LAB website and see data collected and shared to you. Note that data sharing is controlled by the task owner. Upon the consent of task owner, you can click the task of your interest at the dashboard and go to the task page. In the task page, all the task rounds will be displayed in a reverse chronological order. You can view data by clicking the round and data of your interest.

5. Dashboard

5.1 Task Panel

After logging into our website and get access to dashboard, you will see all the available tasks and a brief summary of our collected data as shown below.

5.2 View Task Data

Click the taskname on the task panel, you can check the detailed data statistic of each task app. For example, the below screenshot is the one for MMLABv2.

You can view taskround, cellular logs, task logs and user contribution in the task page.

5.3 Testbed Management

For both task owner and client contributor, you could use testbed management to monitor or even control your device, which would help facilitate the task running and data collection process, on the premises that the advanced options have to be enabled on client app setting. Please refer to this page for client app setting.

5.3.1 Overview

This is the page shown after clicking on “testbed management” on the dashboard. It will show a brief summary and status of all the devices registered to MI-LAB.

5.3.2 Device page

If your deviced is registerd to MI-LAB (set in client app setting), you could find your deivce on the latest devices panel, where you could click on the device ID and get detailed information of your device.

You could also monitor your device status on the bottom of the device page, where you can check if your device is running correctly or if an error occurs.

If you have enabled the remote control option on your client app, you can use the remote control options as shown below to request information from your device or perform several operations. Note that only one operation will be executed on client app at a time.

6. MI-LAB Client App

6.1 Get started

After installing our client app, you should agree to our “Privacy & Terms at the first time. Please check the privacy terms carefully. After that, the client app will redirect you to the home page (shown below)

On the bottom of home page, there are four buttons, which allow you to perform several operations.

6.1.1 Log in

You could use the account created on our website to log in to client app. Or if you have not had one, you could also register one on our client app.

6.1.2 Upload

After finishing running tasks, you could go back to home page and click on the upload button to upload all the logs collected to our server. Note that if you log in as guest, you will not be able to see the logs you collected on our website. Also, you could only check on the logs that are generated by you. Thus, we encourage you to create an account and log in to it whenever you are using our services.

6.1.3 Refresh

By clicking on the button, you will be able to get the latest list of task apps that are distributed by us. As MI-LAB is an open project, task owners may create their own tasks and publish them through our website at any time. Thus, it is encouraged that you refresh the task list from time to time in order to be fully involved and contribute to this project.

6.1.4 Menu

By clicking on this button, you will be able to view the privacy policy and basic information about MI-LAB project. Moreover, you could go into the setting page and configure your device.

6.1.5 Client settings

In the setting page, the device configuration has been broken down into three parts: basic, advanced and dev team (MI-LAB develop team).

Basic
For this part of setting options, you could set the storage size on your device (sdcard/MI-LAB) for the logs collected and also set the uploading methods. By default, uploading will only happen when your device is connected to Wi-Fi.
Advanced
The advanced options work with our testbed management on MI-LAB dashboard. The “register device to MI-LAB” option allows you to monitor your device on testbed management. The “enable remote control” option allows you to control your device and perform several operations through testbed management. As for the hearbeat interval option, this allows you to set how often client updates its status to MI-LAB master. As the remote monitor and control rely on client’s messages, it is suggested that the interval be set lower than 30 seconds (15 seconds by default).
Dev Team
This “Test server”option is discouraged for everyone except MI-LAB develop team members. As MI-LAB is an ongoing project, we keep improving our services and functions. This option will allow our team members to develop and test the newly developed functions without affecting public use. If you accidentally enable this option, you could easily disable it and connect back to MI-LAB server.

6.2 Task operations

You can click one task app and go to the task page (shown below)

6.2.1 Download and install the task app

For the first time, you need to download and install the task app by clicking the left download button. This button allows you to download the latest version of this task app. Whenever you enter the task page, you could read through the text and check if you have the latest task app or not.

6.2.2 Run a task

You can click the right run button to run a task. This button allows you to start MobileInsight plugin of the task first and then then the task app.

Tag: You are encouraged to put a meaningful tag in the customized tag field before you stop the task. This tag will not affect data collection and processing. It is helpful to the task owner and you to understand the collected data if the tag contains such information.
Start MobileInsight: In cases where MobileInsight may not be working properly, MobileInsight button will appear on the task page, which will allow you to start MobileInsight alone again.

Stop task: If you decide to stop the running task, you could go back to client app and press the stop button to wrap up the task properly and collect the generated logs.

Note: the device may shut down suddenly when the battery level is too low (7% observed in our our experience). It is suggested that you stop the task or charge your device when battery level falls below 15%.