Arm Community
Arm Community
  • Site
  • User
  • Site
  • Search
  • User
  • Groups
    • Arm Research
    • DesignStart
    • Education Hub
    • Innovation
    • Open Source Software and Platforms
  • Forums
    • AI and ML forum
    • Architectures and Processors forum
    • Arm Development Platforms forum
    • Arm Development Studio forum
    • Arm Virtual Hardware forum
    • Automotive forum
    • Compilers and Libraries forum
    • Graphics, Gaming, and VR forum
    • High Performance Computing (HPC) forum
    • Infrastructure Solutions forum
    • Internet of Things (IoT) forum
    • Keil forum
    • Morello Forum
    • Operating Systems forum
    • SoC Design and Simulation forum
    • 中文社区论区
  • Blogs
    • AI and ML blog
    • Announcements
    • Architectures and Processors blog
    • Automotive blog
    • Graphics, Gaming, and VR blog
    • High Performance Computing (HPC) blog
    • Infrastructure Solutions blog
    • Innovation blog
    • Internet of Things (IoT) blog
    • Mobile blog
    • Operating Systems blog
    • Research Articles
    • SoC Design and Simulation blog
    • Smart Homes
    • Tools, Software and IDEs blog
    • Works on Arm blog
    • 中文社区博客
  • Support
    • Open a support case
    • Documentation
    • Downloads
    • Training
    • Arm Approved program
    • Arm Design Reviews
  • Community Help
  • More
  • Cancel
Arm Community blogs
Arm Community blogs
Infrastructure Solutions blog Edge Delta on Arm
  • Blogs
  • Mentions
  • Sub-Groups
  • Tags
  • Jump...
  • Cancel
More blogs in Arm Community blogs
  • AI and ML blog

  • Announcements

  • Architectures and Processors blog

  • Automotive blog

  • Embedded blog

  • Graphics, Gaming, and VR blog

  • High Performance Computing (HPC) blog

  • Infrastructure Solutions blog

  • Internet of Things (IoT) blog

  • Operating Systems blog

  • SoC Design and Simulation blog

  • Tools, Software and IDEs blog

Tags
  • Machine Learning (ML)
  • Cloud Native
  • Edge Computing
  • infrastructure
Actions
  • RSS
  • More
  • Cancel
Related blog posts
Related forum threads

Edge Delta on Arm

Julio Suarez
Julio Suarez
November 3, 2021

In this post, we explore running Edge Delta on an Arm-based Jetson Xavier NX at the edge. We demonstrate how Edge Delta can be used to analyze data on an edge device to minimize the amount of data sent to the cloud. We also demonstrate how to extract metrics from the edge device and send them to the Edge Delta admin GUI for visualization.

About Edge Delta

Edge Delta gives enterprises distributed stream processing, federated machine learning, and automated observability data pipeline capabilities. This empowers DevOps, Security, and SRE teams to consume and automatically route all forms of optimized machine data output (logs, metrics, traces). And it provides them with the scalability and efficiency to eliminate technical and financial limitations. This results in organizations that can identify and remediate operational and security incidents more accurately, orders of magnitude faster, and more securely than previously possible with traditional centralized systems.

Use case description

Edge Delta on Arm setup.

Edge side description

On the edge side of this demo, we have the Jetson board which runs an Ubuntu 18.04 image with the Jetpack SDK installed. The Jetpack SDK allows easier use of the CUDA and Tensor cores that are in the Jetson GPU. We use these GPUs to accelerate Machine Learning (ML) inference. To do the inference, we installed the Nvidia Triton Inference server on the Jetson. Triton is a tool that hosts ML models for clients to use. A client (local or remote) can submit inference requests to Triton using HTTP or gRPC (we used gRPC). The request includes data like which ML model to use along with the required input tensors. Triton processes the request with GPU acceleration and sends the results back to the client. In our case, we used Triton to run an object detection model on video frames captured from an attached camera. The model we used was the ssd_mobilenet_v2_2 model that can be found on Tensor Flow Hub. This model can detect many everyday objects within video frames. To keep this post simple, our object detection client code limited its reporting to the number of people the camera sees within it's field of view. Aside from the camera, we also have an integrated temperature/pressure/humidity sensor attached to the Jetson. To keep this post simple, we only use the temperature data from this sensor.

About the ML portion of this demo

We used the Arm Research group's work on the SMARTER project as the basis for the object detection code in this demo. More specifically, we borrowed and modified the image-detector code from the SMARTER project repo. It is not necessary to modify this code to get it to work with Edge Delta, but we modified it since we use this code in other experiments. The main difference in our version of this code is the removal of some unneeded features and some refactoring. And we switched out the object detection ML model with the ssd_mobilenet_v2_2 model as noted earlier. We would like to encourage readers to explore the SMARTER project as it is a very complete example of an edge and cloud use case.

Cloud side description

The cloud side of the use case is straight forward. Edge Delta is a hosted service so all that is needed is an account. Free trials are offered which is helpful for experimentation and exploration purposes. Once an account is created, you can sign in to the Admin GUI. From this GUI, you can obtain instructions and commands on installing the Edge Delta agent on edge devices. In our case, we need only a single command to install the agent on the Jetson. At this point, we are ready to configure the Edge Delta agent, but first we must understand what data we would like to analyze.

Data sources for Edge Delta

Edge Delta supports various types of input data sources. In this demo we use two types of input data sources, files and execs (scripts). Overall, the demo uses four data sources in all.

  • Camera object detection data (Input type: file)
    • Written to /var/log/object_data on the Jetson.
    • The image client code writes the number of people the camera sees in its field of view into this file.
    • The frame rate at which the Jetson can do person analysis is about one frame every 1-2 seconds.
  • Environmental sensor data (Input type: file)
    • Written to /var/log/sensor_data on the Jetson.
    • The sensor client code writes the ambient temperature of the environment the Jetson is in into this file.
    • Rate of temperature samples is about one every second.
  • Authorization logs (Input type: file)
    • Written to /var/log/auth.log by Ubuntu services.
    • This is a log file that reports authorization attempt information. For example, this file logs when a user attempts to SSH into the device.
  • Triton server metrics (Input type: execs)
    • Triton exposes Prometheus style metrics on inferences executed. By default, Triton exposes this on port 8002.

The admin GUI also provides information on how much data has been analyzed at the edge as well as streamed to other services. In our case, we did not integrate Edge Delta with other services and tools to keep things simple (no destinations to stream data to in the following image). We recommend reviewing the documentation on integrations to learn more about this feature.

Data sources flow into Edge Delta agent.

Edge Delta configuration

We do not go into detail on all Edge Delta configuration options, only what is needed to understand how this demo was deployed. For more information on configuring an Edge Delta agent, we suggest reviewing their documentation.

In the configuration file, we define input data sources and processors. The processors can be used to manipulate and extract data from the inputs. Also, processors can search for patterns and anomalies in the data and alert admins when anomalies are found. Edge Delta's ability to process and analyze data locally without having to send and store it all in the cloud is one of its key benefits. The complete configuration we used follows; it is separated into its logical components so that the configurations can be described more easily.

Agent settings

Agent settings are for the agent that is running on the Jetson.

version: v2

agent_settings:
  tag: arm_jetson_demo
  log:
    level: info
  anomaly_capture_size: 1000
  anomaly_confidence_period: 30m

This is mostly default; we just changed the tag to arm_jetson_demo.

Filter configuration

Edge Delta allows for the definition of filters. One of the uses of filters is to remove extraneous information from input data to ease the job of processors. The following code filter removes input data lines that start with a hashtag. The name of the filter is clean_triton_log. We reference this filter further down in the configuration.

filters:
  - name: clean_triton_log
    type: regex
    pattern: "^#.*"
    negate: true

Data input configuration

Below we select the input data sources.

inputs:

  execs:
    - labels: "triton_metrics"
      command: "curl localhost:8002/metrics"
      interval: 1m
      filters: 
        - clean_triton_log

  files:
    - labels: "system_logs, auth"
      path: "/var/log/auth.log"
    - labels: "env_sensors"
      path: "/var/log/sensor_data"
    - labels: "obj_detections"
      path: "/var/log/object_data"

The first data source we see is triton_metrics. To obtain the Triton metrics, we used an executable (execs) input type. This allows us to run the curl command on localhost port 8002. We use localhost because this configuration is for the agent that is running locally on the Jetson alongside the Triton server. Notice that for the execs, we apply the filter called clean_triton_log that we described earlier. This removes lines that start with a hashtag which we know contains no useful data. The last input sources are the log files we described earlier. Note that env_sensors is the label for the temperature sensor input data, we reference this further down in the configuration.

Processor configuration

The following is our processor configuration. These processors are what tell Edge Delta what to do with input data.

processors:

  cluster:
    name: clustering
    num_of_clusters: 25
    samples_per_cluster: 2
    reporting_frequency: 1m

  regexes:
    - name: "temperature"
      pattern: "\\s+temperature=(?P<F>\\d*\\W?\\d+)"
      trigger_thresholds:
        anomaly_probability_percentage: 95

    - name: "cam"
      pattern: ",\\s+(?P<detected>person)=(?P<count>\\d+)"
      dimensions: ["detected"]
      trigger_thresholds:
        anomaly_probability_percentage: 95

    - name: "ssd_mobilenet_v2_2"
      pattern: "^nv_inference_(?P<metric>.*){model=\"ssd_mobilenet_v2_2\",version=\"1\"}\\W+(?P<value>\\d*\\.?\\d+)"
      dimensions: ["metric"]
      trigger_thresholds:
        anomaly_probability_percentage: 90

    - name: "auth_failed"
      pattern: "\\b(?:[Aa]uthentication failure|FAILED SU|input_userauth_request: invalid user|Invalid user|Failed publickey|Failed password)\\b"
      trigger_thresholds:
        anomaly_probability_percentage: 95

    - name: "auth_success"
      pattern: "\\b(?:su:|sudo:|sshd:|sshd\\[|pam_unix).*(?:\\b[Aa]ccepted|session opened|to\\b.*\\bon)\\b"
      trigger_thresholds:
        anomaly_probability_percentage: 95

    - name: "auth_root"
      pattern: "\\b(?:sudo|root|su)\\b"
      trigger_thresholds:
        anomaly_probability_percentage: 95

    - name: "auth_su_attempt"
      pattern: "\\b(?:su:|su\\[).*(?:[Aa]uthentication failure|FAILED SU|input_userauth_request: invalid user|Invalid user|Failed publickey|Failed password)\\b"
      trigger_thresholds:
        anomaly_probability_percentage: 95

At the top, we see a cluster processor defined. These are used to find patterns in the input data and to cluster them together. Having this defined here does not mean that clustering will happen. It still must be "called" in a workflow which we will see in the next section. Below the cluster section, we see various regexes processors. These tell Edge Delta how to extract the metrics we care about. We can see regexes for extracting temperature, person counts from the camera (called cam), and Triton metrics (called ssd_mobilenet_v2_2) related to the ssd_mobilenet_v2_2 model. Like the cluster processor, having these regexes defined here does not mean they are being used. We have to "call" them in a workflow.

Connecting inputs and processors with workflows

Workflows must be defined to tie together the various inputs and processors. The following is what those workflows look like for our demo.

workflows:

  edge_env_sensors_workflow:
    input_labels:
      - env_sensors
    processors:
      - clustering
      - temperature

  edge_camera_workflow:
    input_labels:
      - obj_detections
    processors:
      - clustering
      - cam

  edge_triton_workflow:
    input_labels:
      - triton_metrics
    processors:
      - clustering
      - ssd_mobilenet_v2_2

  stats_workflow:
    input_labels:
      - system_stats

  example_workflow:
    input_labels:
      - system_logs
    processors:
      - clustering
      - auth_failed
      - auth_success
      - auth_root
      - auth_su_attempt

Here we see a few different workflows defined. The first is the edge_env_sensors_workflow. The label env_sensors tells the agent that /var/log/sensor_data is used as the data input for this workflow. Just under the input_labels section, we see the processors. Here we "call" the clustering and temperature processors. This tells Edge Delta to search the /var/log/sensor_data file for patterns and anomalies with the clustering processor, and to extract the temperature readings using the temperature processor. We leave the other workflows up to the reader to study.

Edge Delta dashboard

Next, we explore some of the Edge Delta admin GUI while the demo is deployed. We see that pattern and anomaly detection is a key and useful feature of Edge Delta.

Patterns section

The following is an image of the patterns section in the Admin GUI. This is generated for workflows which use the clustering processor.

 Negative patterns graph showing failed SSH logins and inference model failures.

The Negative Patterns graphs show us two interesting things. The first is that it noticed multiple failed SSH logins into the Jetson at 13:43. This is a useful piece of information as it could indicate a security issue. When these failed attempts occurred, we also received an email alerting us about these failed login attempts. That said, we know this is not an issue because we generated this intentionally by purposely entering an invalid password multiple times while trying to SSH into the Jetson. The second thing we notice from the graph is that we consistently see inference request failures coming from the Triton server logs (logs that start with nv_inference_request). What is interesting is that it is detecting these failures on models that are loaded into the Triton server, but not actually being used. For example, vggish is a model that is loaded, but not being used as part of this demo. This is something that would have gone unnoticed had we not configured Edge Delta to check the Triton logs for patterns with the cluster processor. Now we know there might be a Triton server configuration issue that must be investigated. There are other graphs and logs in the patterns section of the admin GUI, but we leave this up to readers to explore further.

Anomalies section

The anomalies section contains a few different graphs. We show one of them in the interest of brevity.

 Wheel chart showing failed SSH login anamaly.

In the previous graph, the outer rings are various events and metrics the agent analyzes. On the left, we see one of those events highlighted in red. This is hard to read, but it says "auth_failed" which are the failed SSH connection attempts we saw in the Negative Patterns section. Edge Delta is (rightly) labeling these failed login attempts as an anomaly that ought to be investigated.

Metrics Section

The metrics section is straight forward. The metrics shown here are extracted with the regexes processors in our configuration file. The following are temperature and person count graphs.

Temperature plotted across time.

 Number of people charted over time.

As we can see, the temperature where the Jetson is located is stable and consistent. As for the person detection graph, we see that the number of people seen by the camera starts at 0 and eventually rises to 3.

Conclusion

Overall, it was very easy to get up and running with Edge Delta. In fact, we spent more time developing the camera/sensor interface code than we did setting up Edge Delta itself. We would like to encourage readers to give Edge Delta a try on Arm based edge devices. Especially for use cases where edge bandwidth might be limited or where there is cost/performance sensitivity to storing and searching data in the cloud.

Try Edge Delta on Arm

Anonymous
Infrastructure Solutions blog
  • Synchronization Overview and Case Study on Arm Architecture

    Ker Liu
    Ker Liu
    In this blog we highlight a new Synchronization Overview and Case Study on Arm Architecture whitepaper. This whitepaper calls attention to some memory model differences between Arm and x86 and offers suggestions…
    • June 27, 2022
  • LF Edge: Bringing the edge to work

    Tina Tsou
    Tina Tsou
    This blog discusses the mission and opportunity for standardizing the edge.
    • June 8, 2022
  • Spark SQL on AWS Graviton2 demonstrates up to 49% performance benefits over x86

    Masoud Koleini
    Masoud Koleini
    In this blog we demonstrate the performance benefits of AWS Graviton2 on Spark SQL for database queries.
    • June 7, 2022