Tableau

Overview

Tableau has transformed the way organizations around the world analyze, visualize, and share data. By integrating Modzy with Tableau, you add interactive analytics directly into your workflows making your AI models and model findings instantly accessible to more users.

All models deployed into Modzy automatically have RESTful API endpoints and integrating the models using either TabPy orTableau’s network-enabled extension framework is as simple as a few lines of code. The bi-directional integration allows you to select a, select your data source and “run” it, sending the data to Modzy for processing, and returning the results back to the Tableau dashboard for visualization. This activity is also tracked within Modzy's job history, providing a detailed audit trail.

Tableau is a collection of products. This extension is built for either Tableau Prep, where you can combine, shape, and clean your data for analysis, or Tableau Desktop to build visual analytics and interactive dashboards. Select the appropriate integration below for use in your own flows and dashboards!

Tableau Prep

This extension will enable Tableau Prep to process unstructured imagery, video, and audio data within your flow using ML and AI models managed by Modzy.

We will work through a simple example of locating and counting vehicles in aerial imagery using Modzy’s Vehicle Detection model.

• To start, download and install Tableau Prep, and install and start TabPy. Next, retrieve your Modzy API Key.

• Create a simple list of source data files you need to process. An easy way to do this is by navigating to the directory of your source data in the terminal and running a command like ls -d -1 "$PWD/"*.tif > file_list.txt

• Within Tableau Prep Builder, click 'Connect to Data' button, select 'Text File' and navigate to the file list generated in the previous step. Under the Field Separator dropdown, select Comma to make sure Tableau parses your list correctly.

• In the IDE of your choice create a new Python script and paste the following code into it:

from modzy import ApiClient
import pandas as pd

def vehicle_detection_prep(df):
    # set up Modzy Client
    API_URL = "https://trial.app.modzy.com/api"
    API_KEY = "<your.api.key>"
    client = ApiClient(base_url=API_URL, api_key=API_KEY)
    model_id = 'l5lhvcgzt4'
    model_version = '0.0.1'

    # Import image file locations from Tableau Dataframe
    input_data = {}
    for image_src in df[df.columns[0]]:
        input_data[str(image_src.split('/')[-1])] = {
            'image': image_src
        }

    # Send data to Modzy for analysis and retrieve results
    job = client.jobs.submit_file(model_id, model_version, input_data)
    results = client.results.block_until_complete(job)   
    
    # Create DataFrame for Tableau
    results_df = format_results_df(results, input_data)
    
    return (results_df)

def get_output_schema():
    return pd.DataFrame({
        'img_name' : prep_string(), 
        'score' : prep_decimal(), 
        'lat' : prep_decimal(), 
        'lon' : prep_decimal(), 
    })


from osgeo import gdal, osr
def format_results_df(results, input_data):
    result_data = []
    for img in results['results']:
        img_path = input_data[img]['image']
        geoImage = gdal.Open(img_path)
        geoData = geoImage.GetGeoTransform()
        
        src_projection =  osr.SpatialReference(wkt=geoImage.GetProjection())
        src_spatial_reference = int(src_projection.GetAttrValue("AUTHORITY", 1))

        if src_spatial_reference != 4326:
            return('Error: Using an incompatible reference system. Must be reprojected before using.')

        for v in results['results'][img]['results.json']['vehicle']:
            cpx = (v['boundingBox']['xmax'] -  v['boundingBox']['xmin'])/2 + v['boundingBox']['xmin']
            cpy = (v['boundingBox']['ymax'] -  v['boundingBox']['ymin'] )/2 + v['boundingBox']['ymin']

            lat = geoData[3] + cpy * geoData[5]
            lon = geoData[0] + cpx * geoData[1]
            
            score = v['score']
            result_data.append([img, score, lat, lon,])

        results_df = pd.DataFrame(result_data, columns = ['img_name','score','lat','lon'])
    return results_df

• In the flow editor, click the blue '+' and select Script. Select Tableau Python (TabPy) Server, and using the Browse button select the script saved in the previous step. Paste vehicle_detection_prep into the Function Name box. Tableau should load the script and display 4 empty columns: score, img_name, lon and lat.

• Click the blue '+' next to the script and select Output. In the options, select the save location, file type and name as desired.

• Click Run Flow. The flow will take some time to execute, usually around 0.5 seconds per source image file. When the output is generated, check to make sure it appears generally correct.

• Open Tableau Desktop to a new book. Using the New Data Source button or menu, connect to the output generated in the previous step.

• Click Sheet 1 in the lower left, then drag Lon to the Columns row and Lat to Rows. Change each to a Dimension in the measure dropdown menu.

• A map of all the datapoints will be generated!

Tableau Desktop

The Modzy Tableau Extension integrates Modzy’s capabilities into Tableau using Tableau’s Extension API. The application consists of Express.js endpoints along with Javascript function that interact with both the Tableau Extension API and the Modzy API. The outputs return a D3.js visualization that allows a user to filter the Tableau dashboard based on Modzy’s returned output.

1. Add handler function in app.js -> run/{modelId} endpoint for the specific model being integrated. (Need to ensure that the raw data from Tableau is transformed and sent in the format that Modzy requires)
2. Add model details into main.js run() function and ensure that visualization being displayed appropriately lines up with the data being returned from Modzy.
3. Add model details into config.html to have the option to select the new model.

In addition to visualizing model outputs, Tableau with Modzy can be used to visualize the entire MLOps lifecycle. From which users and which applications are using which models, to inference time, to model and data drift characterizations, Modzy’s logging and audit API makes it easy to chart and visualize your workflow and ML usage data. Further, tracking Deployment Frequency, Lead Time for Changes,

Using a tool like Modzy you’ll be able to take advantage of your existing applications such as Tableau and easily integrate AI to your business operations, ensuring that the value of your technological investments are fully realized.