News & insights

The rapid spread and destructive nature of wildfires and volcanic activity have intensified the need for low latency detection systems. The growing intensity and frequency of globally distributed thermal hotspots have driven the development of satellite-based detection solutions. Conventional approaches rely on ground-based processing, which limits low latency capabilities due to revisit times over ground stations and data handling requirements. This work proposes the first onboard payload processing pipeline for segmentation of thermal hotspots in raw multispectral satellite imagery. The pipeline leverages the Near InfraRed (NIR) and Short-Wave InfraRed (SWIR) spectral bands, and the combination of onboard Artificial Intelligence (AI) and raw imagery significantly reduces the delay between image acquisition and event detection. Furthermore, we present Segmentation of Thermal Hotspots in Raw Sentinel-2 data (SegTHRawS), the first publicly available dataset for thermal hotspot segmentation in raw multispectral satellite imagery. The segmentation model employed is a Fully Convolutional Network (FCN) derived from U-Net, named ResUnet-S2, designed for fast on-device inference. This model achieved an Intersection over Union (IoU) of 0.988 and an F-1 score of 0.986 on SegTHRawS, with its detection and generalization capabilities validated using an external thermal hotspot segmentation dataset. The proposed pipeline was verified on CubeSat-compatible hardware, achieving an end-to-end execution, from image acquisition to event detection, in 1.45 s, faster than the image acquisition process, and consuming a peak power of 4.05 W. These results demonstrate the potential of onboard processing solutions for minimizing the detection latency of current approaches, particularly for thermal hotspot segmentation, using edge computing satellite hardware.

Dynamic targeting (DT) is a spacecraft autonomy concept in which sensor data is acquired and rapidly analyzed and used to drive subsequent observation. We describe the low Earth orbit application of this approach in which lookahead imagery is analyzed to detect clouds, thermal anomalies, or land use cases to drive higher quality near nadir imaging. Use cases for such a capability include: cloud avoidance, storm hunting, search for planetary boundary layer events, plume study, and beyond. The DT concept requires a lookahead sensor or agility to use a primary sensor in such a mode, edge computing to analyze images rapidly onboard, and a primary followup sensor. Additionally, an inter-satellite or low latency communications link can be leveraged for cross platform tasking. We describe implementation in progress to fly DT in early 2025 on the CogniSAT-6 (Ubotica/Open Cosmos) spacecraft that launched in March 2024 on the SpaceX Transporter-10 launch.

In partnership with Ubotica Technologies, the Jet Propulsion Laboratory is demonstrating state-of-the-art data analysis onboard CogniSAT-6/HAMMER (CS-6). CS-6 is a satellite with a visible and near infrared range hyperspectral instrument and neural network acceleration hardware. Performing data analysis at the edge (e.g. onboard) can enable new Earth science measurements and responses. We will demonstrate data analysis and inference onboard CS-6 for numerous applications using deep learning and spectral analysis algorithms.

The successful launch of the MANTIS mission, on November 11th 2023, was one of the key milestones of this three year long development. MANTIS is a commercial Earth Observation mission targeting the energy and mining industries whose resources are usually found in remote and hostile regions and that require dedicated and high resolution monitoring. The mission is part of the UKSA supported ESA-InCubed programme which stimulates innovation in industry by co-funding industry-initiated projects. The consortium is led by Open Cosmos and includes SATLANTIS as payload provider and Ubotica and Ingeniars as suppliers of the onboard AI capabilities. The space segment is based on a newly designed 12U Cubesat, operating from a 525km-SSO orbit. The spacecraft in its final configuration weights 14.5 kg and delivers 13km-swath images with a native GSD of 3.5m in 4 bands (RGB and NIR). One of the peculiarities of the SATLANTIS iSIM90 is its binocular configuration that allows for fast image acquisition and the application of their proprietary UHR algorithm that enhances the GSD to 2.2/3m in the final super-resolved image. To complete the payload architecture, an onboard AI accelerator applies a cloud detection algorithm able to automatically discard cloudy images enhancing the overall mission efficiency. The satellite is operated with an S-band link and downloads the images through an X-band downlink to ground stations located at Svalbard and TrollSat, Antarctica. The images are stored and processed within Open Cosmos data platform called DataCosmos. The mission has now successfully completed the In Orbit Commissioning Review (IOCR) and is currently calibrating the payload chain. More than 500 GB of data has already been downloaded since the end of Launch and Early Orbit Phase (LEOP). MANTIS will start the operational phase in H2 2024. This paper will briefly introduce the programmatic background, purpose, consortium and development of the MANTIS mission including the technological advancements that are not only related to the payload chain but also to the Earth Observation tailoring of the platform design and its advancements in terms of EPS, OBDH and high speed data link implementation. Focus of the paper will be the results of LEOP together with the IOCR since launch and initial results regarding the mission compliance with its user requirements. Examples of the images captured, related data processing activities and associated data products will be shown to provide a summary of the current system capabilities, including its initial performances. A conclusive section will elaborate on the exploitation for different application domains such as energy, mining, infrastructure or agriculture monitoring.

When an Artificial Intelligence model runs in a real scenario, two situations are possible: 1) the data analysed follows the same distribution as the data used for model training and therefore the model performance is similar; or 2) the distribution of the new data is different, resulting in lower model performance. This is called "data/domain shift" and its measurement is desirable in order to reduce it. In this work we propose using the Jensen-Shannon divergence score to measure the data shift. Results, obtained by using 5-fold cross-validation, show high correlation between the proposed metric and the accuracy when test samples are modified for different brightness, sharpness and blur. The approach has applicability to autonomously measuring domain shift in Earth Observation data.

This work presents the radiation characterization of the COTS Myriad X Vision Processing Unit, from Intel Movidius, for proton- and heavy ion-induced Single Event Effects and Total Ionizing Dose. The component has already flown on different missions, and it continues to be considered in future ones. The results obtained show that the radiation performance of the device is aligned with the performance of other components manufactured in the 16nm FinFET technology. Potential use cases of Myriad X in space computing platforms are also discussed.

Attitude determination is a crucial task for space missions and relies on multiple onboard sensors such as sun sensors, magnetometers, and Earth horizon sensors. Moreover, star trackers, which identify stars in a scene and match them against an existing star catalog to determine the attitude, provide superior performance compared to traditional sensors and they were previously reserved for high-end missions. With the increasing popularity of small satellites, a trade-off between cost, efficiency, and precision is often encountered. Nowadays, star sensors have undergone significant advancements, becoming more efficient and accessible due to notable enhancements in hardware and software, particularly through the integration of neural networks. This leveraging of artificial intelligence (AI) has enabled the development of a compact and reliable star sensor, potentially eliminating the need for other sensor types. In this work, 6-synchronized star-trackers (6SST), a sensor with multiple imaging channels, is proposed to get wider celestial coverage and hence reliability.

Mapping sea ice in the Arctic is essential for maritime navigation, and growing vessel traffic highlights the necessity of the timeliness and accuracy of sea ice charts. The AutoICE Challenge investigates the possibility of creating deep learning models capable of mapping multiple sea ice parameters automatically from spaceborne synthetic aperture radar (SAR) imagery. Participants were tasked with the development of machine learning algorithms mapping the total sea ice concentration, stage of development, and floe size using a state-of-the-art sea ice dataset. The challenge had 129 teams representing a total of 179 participants, with 34 teams delivering 494 submissions, and it was won by a team from the University of Waterloo.

In partnership with Ubotica Technologies, the Jet Propulsion Laboratory is demonstrating state-of-the-art data analysis onboard CogniSAT-6. Capabilities of in-orbit assets to perform Earth science has skyrocketed in recent years. Performing data analysis at the edge (e.g. onboard) can enable new Earth-science measurements and responses. CogniSAT-6/HAMMER (CS-6) is a satellite with a visible and near infrared range hyperspectral instrument and neural network acceleration hardware. We will demonstrate data analysis and inference onboard CS-6 for numerous applications using deep learning and spectral algorithms.

Earth Observation spacecraft play a pivotal role in various critical applications impacting life on Earth. Historically, these systems have adhered to conventional operational paradigms, namely the "mow-the-lawn" and "bent pipe" approaches. We aim to depart from these traditional methodologies by integrating onboard Artificial Intelligence, real-time communication, and new observing strategies in one system called CogniSAT-6. These transformative innovations will amplify the amount, speed, and quality of the information yielded by such a system by up to an order of magnitude. This paper provides an overview of the current state of the art in autonomous Earth Observation spacecraft and the application of onboard processing in Earth Observation spacecraft, an overview of the CogniSAT-6 mission, its concept of operations, system architecture, and data processing design, and an ethical framework for such systems.

In March 2024, the first Intelligent Space Camera (ISC) from Ubotica Technologies was launched on the Apex "Call to Adventure" small satellite mission. The ISC is a designed-for-space camera payload with hardware acceleration of Artificial Intelligence (AI) and Computer Vision (CV) algorithms integrated directly into the camera. Deployable as a self-contained unit within a spacecraft, the ISC manages and executes various processes autonomously and internally without external dependencies, enabling new AI-driven automation-based paradigms and capabilities for space systems. This short paper describes the novel payload, its capabilities, and the mission it is flying on.

Solar Active Regions are characterised by their intense magnetic activity, which often leads to solar phenomena such as solar flares and coronal mass ejections (CMEs). In this study, we propose the use of a Temporal Convolutional Network (TCN) for predicting whether an active region will be flaring in a specific window of time and defining the flare class. The proposed solar flare prediction ensemble (EoFNets) is based on both the physical characteristics of the active region (EoFPhyNet) and geometric features (EoFGeoNet). Experimental results show that TCN outperforms long short-term memory (LSTM) in three cases. The EoFNets network has been optimised to fit the resource constraints of a space platform deployed at the extreme edge far from Earth.

Recent developments in New Space companies have led to a dramatic increase in capabilities in Earth Observation. These advances in edge computing, low latency communications, and many new on orbit assets represent a unique opportunity for Earth Observation. NASA's New Observation System (NOS) program aims to leverage these new capabilities to achieve global reach of science events such as volcanic eruption, wildfires, and flooding — not by wide swath instruments but rather by intelligent, directed sensing, onboard analysis, and dissemination of knowledge rather than data. We describe ongoing efforts to deploy NOS capabilities to the CogniSAT-6/HAMMER satellite launched in March 2024.

In recent years, Deep Neural Networks (DNNs) approaches have outperformed traditional techniques for several computer vision problems. In this context, we examine several methods for noise reduction and star detection for improving centroiding performance. An efficient and robust denoising method for star images using an Auto-Encoder (AE) is proposed, along with an accurate and lightweight algorithm based on an existing YOLO architecture to detect the location of stars in the image. An extensive comparison of the proposed denoising technique with other traditional filters confirms that the proposed method resists all noise models and reconstructs well the corrupted images.

This study explores the development and implementation of machine learning (ML) models on space-qualified AI boards aiming to identify the most effective solution for implementing anomaly detection systems on space missions. We investigate various ML anomaly detection techniques including Autoencoders, Long Short-Term Memory (LSTM) cells, Isolation Forests and Transformers. These models were trained on a univariate dataset derived from real space missions and deployed on hardware engineered for space environments. The paper provides a detailed examination of both the optimal ML models and hardware platforms for deploying univariate anomaly detection systems in space flight contexts.

This paper describes a novel approach for performing geolocation of insights extracted on-board satellites. Neural networks, after a training process, are able to compress raw imagery into numerical vectors, referred to as embeddings, which are much smaller while keeping the most relevant information. These embeddings are small enough to be transmitted over always-available Inter Satellite Links alongside insights extracted on-board, thereby decoupling their transmission from the overpass of ground stations, reducing latency to delivery. The developed neural network is able to capture the most notable features contained within the provided image, providing results with an absolute error lower than 20 meters for a variety of regions.

This paper presents "the OPS-SAT case", a novel data-centric competition that seeks to address the challenge of high-quality annotated data for training AI systems in space. The powerful computational capabilities of the European Space Agency's OPS-SAT satellite are utilized to showcase the design of machine learning systems for space by using only the small amount of available labeled data. The generation of a suitable dataset, design and evaluation of a public data-centric competition, and results of an onboard experimental campaign using the competition winners' machine learning model directly on OPS-SAT are detailed.

Artificial intelligence (AI) has increasingly found its way into various space applications, with edge AI proving particularly useful in certain scenarios. One application of edge AI in space is solar observation satellites. The ESA VIGIL Mission is a space weather mission that will monitor the Sun from the L5 Lagrange point, and the first of its kind to do it operationally from that position. For the VIGIL mission, ESA has developed a dedicated computational memory space grade hardware and an Artificial Intelligence ensemble system tool to automatically classify solar flares. The system has demonstrated promising accuracy and performance in its first results, and it has shown the ability to detect CMEs and predict the possibility of CME occurrence from active areas on the Sun.

Future space missions can benefit from processing imagery onboard to detect science events, create insights, and respond autonomously. Modern Commercial Off The Shelf (COTS) processors, such as the Movidius Myriad X and the Qualcomm Snapdragon, provide significant improvements in small Size Weight and Power (SWaP) packaging and offer direct hardware acceleration for deep neural networks. Neural network models are deployed on these processors hosted by Hewlett Packard Enterprise's Spaceborne Computer-2 onboard the International Space Station (ISS). The Myriad and Snapdragon DSP/AIP provide speed improvement over the Snapdragon CPU in all cases except single pixel networks (typically >10x for DSP/AIP).

To enable the Earth Observation space systems required to serve the needs of life on Earth in the near future, these systems need to operate more efficiently and autonomously. Artificial Intelligence can be deployed on the edge on spacecraft to provide this required increased autonomy. CogniSAT-6, an upcoming CubeSat Earth Observation mission by Ubotica and Open Cosmos, will leverage this technology to interpret captured images and use this extracted information to autonomously schedule operations without any input from ground. This capability greatly increases the efficiency of Earth Observation systems and enables tip-and-cue scenarios.

Building Artificial Intelligence (AI) models for Earth Observation (EO) satellites can be a difficult task since there is often no, or very limited, real training data available for a particular sensor. The main purpose of this work is to investigate and develop how to update remotely a model placed in a satellite in orbit, and how to control or specify the update size using the training parameters. The method proposed selects which Convolutional Neural Network (CNN) layer weights must be modified and which must be fixed during training in order to maximize the accuracy increase and minimize the update file size. With the proposed Efficient Network Update (ENU) method, the generated post-training network update is only 18MB in size, but still achieves an accuracy of 78.9%.

The Φsat-2 mission from the European Space Agency (ESA) is part of Φsat mission lineup aimed to address innovative mission concepts making use of advanced onboard processing including Artificial Intelligence. Φsat-2 is based on a 6U CubeSat with a medium-high resolution VIS/NIR multispectral payload combined with a hardware accelerated unit capable of running several AI applications throughout the mission lifetime. The following applications are currently baselined: SAT2MAP for autonomous detection of streets during emergency scenarios; Cloud Detection application and service for data reduction; the Autonomous Vessel Awareness to detect and classify vessel types; and the deep compression application (CAE) to reduce the amount of acquired data.

The Intelligent Space Camera (ISC) is a compact space camera with embedded Computer Vision and Artificial Intelligence capabilities, that can address applications requiring high-throughput smart processing directly at source. The camera, incorporating both hardware and software elements, is being developed in the frame of an ESA co-funded project to support space situational awareness, visual FDIR, and docking applications, among others. The camera, built around the Myriad X Vision Processing Unit (VPU), supports RTSP streaming, H.265 encoding, dynamic remote reconfiguration, and in-line AI stream processing at framerate, all directly on-camera.

In this paper, an Enhanced CSD (3CSD) is introduced. An efficient and flexible storage device-based triage, that provides a seamless workflow to process and infer data in place represents an evolution in terms of traditional CSD which employs a store-first inference-later paradigm. The experimental results show that the purpose of Multistage LILLIAN (LeveragIng Last-miLe data usIng contextbased AutolabelliNg) increases the performance of the triage detectors by 20%. Furthermore, some state-of-the-art fusion approaches are evaluated including late fusion (Ensembles) which boosted the performance of the triage subsystem significantly. Experimental results are evaluated using Intel Movidius Myriad X.

As part of an initiative to promote the development and implementation of innovative technologies on-board Earth Observation (EO) missions, ESA kicked off the first Φsat related activities in 2018. The selected Φsat-2 concept will provide a combination of on-board processing capabilities (including AI) and a medium to high resolution multispectral instrument from Visible to Near Infra-Red (VIS/NIR). These resources will be made available to a series of dedicated applications that will run on-board the spacecraft. AI applications under development include SAT2MAP for autonomous street detection, an enhanced cloud detection experiment, Autonomous Vessel Awareness, and a Deep Compression application. The Φsat-2 mission successfully passed the CDR phase at the end of 2022, aiming for a launch in 2024.

The objective of the AIVIONIC technology development project is to implement a HW/SW demonstrator of an AI-based Visual Navigation System. Neural Network (NN)-based algorithms were identified considering specific mission characteristics such as on-board implementability and algorithm adaptability and flexibility. Lightweight, modular AI processing pipelines were selected and implemented, employing Object Detection and Keypoint Regression Networks. The HW platform for the AIVIONIC visual navigation system is composed of Ubotica's CogniSat ecosystem, based on the Intel family of Myriad Vision Processing Units, together with the Xilinx Zynq UltraScale+. The obtained results show that the objective of developing a HW/SW demonstrator for a vision-based relative navigation system using AI in a dependable manner has been achieved, reaching TRL 4.

We benchmark deep learning models on Intel’s Movidius Myriad X Vision Processing Unit, enabled by HPE’s Spaceborne Computer-2. Deployed classifiers include neural networks trained on Mars imagery from the Reconnaissance Orbiter and Curiosity rover as well as a Myriad X memory test. This work is a step towards running similar classifiers for a range of applications, including onboard autonomy and data distillation.

We benchmark deep learning models on Intel’s Movidius Myriad X Vision Processing Unit, enabled by HPE’s Spaceborne Computer-2. Deployed classifiers include neural networks trained on Mars imagery from the Reconnaissance Orbiter and Curiosity rover as well as a Myriad X memory test. This work is a step towards running similar classifiers for a range of applications, including onboard autonomy and data distillation.

This study evaluates a convolutional neural network (CNN)-based pose estimation system for real-time relative navigation with uncooperative spacecraft, leveraging Intel’s Myriad X Vision Processing Unit (VPU) on Ubotica’s CogniSAT CubeSat board. The research explores adapting the Pose HRNet model, initially trained on GPUs, to operate effectively on the Myriad X—a low-power processing platform designed for small satellites. Key findings reveal the model achieves comparable accuracy and performance on the VPU, with a significantly higher inference efficiency per watt, critical for power-constrained space environments. The study validates the full processing pipeline in an orbital context, demonstrating the potential for deploying onboard pose estimation capabilities in future small satellite missions.

With the adoption of edge AI processors for space, on-orbit inference on EO data has become a possibility. Since the development of on-orbit AI applications requires rarely available raw data, training of these AI networks remains a challenge. We investigate the effects of varying two key image parameters between training and testing data on a ship segmentation network: Ground Sampling Distance and band misalignment magnitude. Our results show that for both parameters the network exhibits degraded performance if these parameters differ in testing data with respect to training data, and that this performance drop can be mitigated with appropriate data augmentation.

Future space missions can benefit from processing imagery onboard to detect science events, create insights, and respond autonomously. Modern COTS processors, such as the Movidius Myriad X and the Qualcomm Snapdragon, provide significant improvements in small Size Weight and Power (SWaP) packaging and offer direct hardware acceleration for deep neural networks. Neural network models are deployed on processors hosted by HPE's Spaceborne Computer-2 onboard the ISS. A variety of algorithms are benchmarked, and the models are run multiple times on the ISS to detect any errors. A memory checker is also run to detect radiation effects on the embedded processors.

Artificial intelligence (AI) is paving the way for a new era of algorithms focusing directly on the information contained in the data, autonomously extracting relevant features for a given application. Recent advances in microelectronics provide hardware accelerators with an efficient ratio between computation and energy consumption, enabling the implementation of AI algorithms at the edge. The Φ-sat-1 has demonstrated the potential of artificial intelligence (AI) as a reliable and accurate tool for cloud detection on-board a hyperspectral imaging mission. Activities included demonstrating the robustness of the Intel Movidius Myriad 2 hardware accelerator against ionizing radiation, developing a Cloudscout segmentation neural network run on Myriad 2, and assessing the innovative Hyperscout-2 hyperspectral sensor. This mission represents the first official attempt to successfully run an AI deep convolutional neural network directly inferencing on a dedicated accelerator on-board a satellite.

Neural network models are benchmarked on Intel's Movidius Myriad X Vision Processing Unit, enabled by HPE's Spaceborne Computer-2. Deployed classifiers include neural networks trained on Mars imagery from the Reconnaissance Orbiter and Curiosity rover as well as a Myriad X memory test. This work is a step towards running similar classifiers for a range of applications, including onboard autonomy and data distillation.

φ-Sat-1 is part of the European Space Agency initiative to promote the development of disruptive innovative technology and capabilities on-board EO missions. The φ-Sat-1 satellite represents the first-ever on-board Artificial Intelligence (AI) deep Convolutional Neural Network (CNN) inference on a dedicated chip attempting to exploit artificial Deep Neural Network (DNN) capability for Earth Observation. It utilises the Myriad Vision Processing Unit (VPU), a System On Chip (SOC) that has been designed ex novo for high-performance edge compute for vision applications. In order to support Myriad’s deployment on φ-Sat-1, the first mission using AI processing for operational purposes, and future applications in general, the SOC has undergone radiation characterisation via several test campaigns in European test facilities. The first AI application developed for in-flight inference was CloudScout, a segmentation neural network that was designed specifically for φ-Sat-1 in order to achieve high detail and good granularity in the classification result, and eventually discard on-board the cloudy images acquired by the hyperspectral sensor, thus greatly enhancing the data throughput capability of the mission. In addition to the CloudScout cloud detection AI SW results acquired during φ-Sat-1’s mission, in-flight performance data was also acquired for the hardware inference engine. Four separate VPU-based inference engine test phases were executed over 70 days during the mission. The in-flight diagnostics tests for the VPU inference engine indicate that the device performed as expected on-board φ-Sat-1 without experiencing any functional upsets, or any functional degradation effects due to radiation. All future installations of the Myriad VPU in space will be equipped with this Built-In Self Test (BIST) that will allow monitoring the performance of the inference engine hardware.

Deep Space missions can benefit from onboard image analysis. Traditional space flight hardware provides modest compute when compared to today's laptop and desktop computers. New generations of commercial off the shelf (COTS) processors, such as the Qualcomm Snapdragon and Movidius Myriad X, deliver significant compute in small Size Weight and Power (SWaP) packaging and offer direct hardware acceleration for deep neural networks. Neural network models are deployed on these processors hosted by Hewlett Packard Enterprise's Spaceborne Computer-2 onboard the International Space Station (ISS). A variety of algorithms trained on imagery from Earth or Mars, as well as some standard deep learning models for image classification, are benchmarked.

We benchmark Intel’s Movidius Myriad X Vision Processing Unit, enabled by HPE’s Spaceborne Computer-2. Deployed classifiers include neural networks trained on Mars imagery from the Reconnaissance Orbiter and Curiosity rover. This work is a step towards running similar classifiers for a range of applications, including onboard autonomy and data distillation.

This technical paper describes the High-Performance Compute Board (HPCB), currently being implemented and tested by a consortium led by Cobham Gaisler in the frame of an ESA project. The first section serves as a brief introduction to the platform, whereas subsequent sections add further detail concerning the architecture, hardware, and software design. Finally, some preliminary test results are presented before summarizing the most relevant aspects of the paper in the conclusions.

The advent of computationally demanding algorithms and high data rate instruments in new space applications pushes the space industry to explore disruptive solutions for onboard data processing. We examine heterogeneous computing architectures involving high-performance and low-power commercial SoCs. The current paper implements an FPGA with VPU co-processing architecture utilizing the CIF & LCD interfaces for I/O data transfers. A Kintex FPGA serves as our framing processor and heritage accelerator, while we offload novel DSP/AI functions to a Myriad2 VPU. We prototype our architecture in the lab to evaluate the interfaces, the FPGA resource utilization, the VPU computational throughput, as well as the entire data handling system’s performance, via custom benchmarking.

The market for remote sensing space-based applications is fundamentally limited by up- and downlink bandwidth and onboard compute capability for space data handling systems. This article details how the compute capability on these platforms can be vastly increased by leveraging emerging commercial off-the-shelf (COTS) system-on-chip (SoC) technologies. The orders of magnitude increase in processing power can then be applied to consuming data at source rather than on the ground allowing the deployment of value-added applications in space, which consume a tiny fraction of the downlink bandwidth that would be otherwise required. The proposed solution has the potential to revolutionize Earth observation (EO) and other remote sensing applications, reducing the time and cost to deploy new added value services to space by a great extent compared with the state of the art. This article also reports the first results in radiation tolerance and power/performance of these COTS SoCs for space-based applications and maps the trajectory toward low Earth orbit trials and the complete life-cycle for space-based artificial intelligence classifiers on orbital platforms and spacecraft.

Star trackers are navigation sensors that are used for attitude determination of a satellite relative to certain stars. A star tracker is required to be accurate and also consume as little power as possible in order to be used in small satellites. While traditional approaches use lookup tables for identifying stars, the latest advances in star tracking use neural networks for automatic star identification. This manuscript evaluates two low-cost processors capable of running a star identification neural network, the Intel Movidius Myriad 2 Vision Processing Unit (VPU) and the STM32 Microcontroller. The intention of this manuscript is to compare the accuracy and power usage to evaluate the suitability of each device for use in a star tracker. The Myriad 2 VPU and the STM32 Microcontroller have been specifically chosen because of their performance on computer vision algorithms alongside being cost-effective and low power consuming devices. The experimental results showed that the Myriad 2 proved to be efficient and consumed around 1 Watt of power while maintaining 99.08% accuracy with an input including false stars. Comparatively the STM32 was able to deliver comparable accuracy (99.07%) and power measurement results. The proposed experimental setup is beneficial for small spacecraft missions that require low-cost and low power consuming star trackers.

The historic success of the recent ESA Φ-sat-1 mission has demonstrated for the first time that COTS hardware acceleration of AI inference on a satellite payload in-orbit is now possible. The Deep Learning cloud detection solution deployed on Φ-sat-1 utilises an Intel Movidius Myriad 2 vision processor for inference compute. The Myriad has performance-per-watt and radiation characteristics that make it ideally suited as a payload data processor for satellite deployments, providing state-of-the-art Neural Network (NN) compute within an industry-low power envelope. Building on the hardware and software deployed on Φ-sat-1, the UB0100 CubeSat board is the next generation AI inference and Computer Vision (CV) engine that addresses the form factor and interface needs of CubeSats while exposing the compute of Myriad to the payload developer. This presentation discusses the requirements of an AI CubeSat payload data processing board (hardware, firmware, software), and demonstrates how the UB0100 solution addresses these requirements through its custom CubeSat build. An overview of the CVAI software that runs on the UB0100 will show how, in addition to AI inference and integration with popular AI frameworks, the user now has direct access to the hardware-accelerated vision functionality of the Myriad VPU. This unlocks combined image pre-processing and AI compute on a single device, enabling direct processing of data products at different levels on-satellite. The flexibility provided to the user by the UB0100 solution will be demonstrated through a selection of use cases.

Computer vision and deep learning are clearly demonstrating a capability to create engaging cognitive applications and services. The Eyes of Things (EoT) is a powerful and versatile embedded computer vision platform which allows the user to develop artificial vision and deep learning applications that analyse images locally. In this article, the deep learning capabilities of an EoT device are applied to a real-life facial informatics application: a doll capable of recognizing emotions using deep learning techniques and acting accordingly. The main impact is in showing that a toy can now do advanced processing locally, without the need of cloud computation, thus reducing latency and removing most of the ethical issues involved.

Embedded systems control and monitor a great deal of our reality. This paper describes the Eyes of Things (EoT) platform, a versatile computer vision platform designed to bring autonomous intelligence to embedded devices. While not everything needs computer vision, visual information is the richest source of information about the real world: people, places and things. The EoT platform addresses the challenges and opportunities of embedded computer vision, including new applications and technologies such as deep learning, drones, home robotics, intelligent surveillance, intelligent toys, wearable cameras, and more.