Advances in Machine Learning, Reliability, and Signal Processing

Matthew Avery Chair
 
Tuesday, Aug 8: 8:30 AM - 10:20 AM
0065 
Contributed Papers 
Metro Toronto Convention Centre 
Room: CC-203B 

Main Sponsor

Section on Statistics in Defense and National Security

Presentations

Bayesian Sustainment and Fleet Management Utilizing Ontology Driven Design

Management and sustainment of large fleets is a key driver in cost reduction within the defense industry. The industry is comprised of many fleets which require a common solution for all similar systems to avoid siloed products. Additionally, many fleets involve diverse resources including people, machines, parts, etc. demanding a solution that maps data and domain knowledge in a flexible way.
Northrop Grumman addresses this with a knowledge driven data architecture comprised of reusable microservices and fleet management products, enabling the extraction and modeling of core structures consistent across all fleets while enhancing scalability. The solution ingests expert knowledge and complex data sources into its ontological data model and standardizes prognostic methods ranging from Bayesian inference to modern machine learning.
In this paper we discuss the fundamental data elements and design principles required to model an abstract fleet. We also discuss how the solution maps data onto these abstract objects–hydrating an ontological domain. Finally, we address techniques used to enable the prognostic model abstraction needed to feed a data driven fleet sustainment solution. 

Keywords

Bayesian Inference

Ontology Driven Design

Fleet Management

Fleet Sustainment

Reliability 

View Abstract 3243

Co-Author

McKay Davis, Northrop Grumman

Presenting Author

Joel Linford, Northrop Grumman

Enforcing hierarchical label structure for multiple object detection in remote sensing imagery

Automatic object detection in remote sensing imagery flags objects of interest in a scene. We are interested in multi-label classification of images in the FAIR1M benchmark dataset, which contains "ground truth" images labeled with 5 coarse and 37 fine hierarchical object classes summarizing scene content. We propose using a SwinV2 visual backbone that feeds into a transformer to produce multi-label taxonomic classification. The flexibility of deep learning models for computer vision makes it possible to identify targets that would otherwise be difficult to explicitly quantify. However, with naïve application, the classifications from a multi-task model may not uphold known hierarchical structure. Common approaches to acknowledge hierarchical structure of labels include adding penalties for inconsistent predictions in the cost function, using different features to predict coarse/fine classes, and completing classification tasks of different granularities at different depths in the model. We implement the proposed model with additional hierarchy aware modifications and compare to naïve flat classification. 

Keywords

deep learning

hierarchical

computer vision

remote sensing

penalization

classification 

View Abstract 3517

Co-Author(s)

Cindy Gonzales, Lawrence Livermore National Laboratory
Wesam Sakla, Lawrence Livermore National Laboratory

First Author

Laura Wendelberger, Lawrence Livermore National Laboratory

Presenting Author

Laura Wendelberger, Lawrence Livermore National Laboratory

A Sequential Monte Carlo Library Built on TensorFlow for Modeling System Reliability

As the Department of Defense (DoD) aims to become more efficient, condition-based maintenance (CBM) is an expected support capability of modern systems. Physics-driven state-space models are often selected to address CBM on highly reliable systems due to the method's ability to leverage state observations and highly accelerated life testing (HALT). However, such models are time intensive to develop and computationally demanding to simulate. In this paper, we present a library extending TensorFlow for component modeling and probabilistic prediction of Remaining Useful Life (RUL) to inform CBM through physics-driven state-space models. This library provides sufficient generality and application of finite difference, sequential Monte Carlo filters, Monte Carlo (MC) simulation, and Markov chain Monte Carlo (MCMC) methods allowing for a consistently deployable framework with the option to tailor elements of the implementation as needed. We show how the library architecture enables translation of telemetry and HALT data to RUL prediction and model deployment on a production architecture enabling optimized CBM on DoD systems. 

Keywords

Bayesian

Remaining useful life

sequential Monte Carlo

Reliability

conditioned based maintenance

control theory 

View Abstract 3239

Co-Author(s)

Zach Tretter, Northrop Grumman
Joel Linford, Northrop Grumman

First Author

Michael Snyder, Northrop Grumman

Presenting Author

Michael Snyder, Northrop Grumman

Machine Learning-Statistics Ensemble Battery EOL Prediction Model

Prediction of Lithium-ion battery end-of-life (EOL, commonly defined as capacity dropping below 80% of nominal) is a condition-based maintenance (CBM) problem with wide application in industry. A joint team from MIT and Stanford conducted highly accelerated life testing (HALT) experiments on 124 LFP/graphite batteries; they and others have generated additional features and constructed EOL models, identifying particularly predictive features and achieving high accuracies. In this work, we examine the effects of adding more features, fitting data to fundamental functional forms, and applying routine statistical operations and other relevant methods on the accuracy of the resulting ensemble model in predicting EOL early in cycle life. 

Keywords

Machine Learning

Statistics

Finite Difference

Ensemble

Battery

EOL Prediction Model 

View Abstract 3242

Co-Author

Michael Snyder, Northrop Grumman

First Author

Brian Hansen

Presenting Author

Brian Hansen

Statistical modeling for detecting anomalous event timings on host-based log data

U.S. networks are constantly attacked by a variety of advanced tactics that evade existing detection techniques, leading to attack compromises. To this end, we seek an accurate and timely detection of host-based attacks targeting U.S. enterprise security and critical infrastructure. Our approach uses host-based data logs (e.g. Sysmon), which describe events that occur on computers over time, and are an important tool enabling cyber analysts to detect anomalous and/or malicious event sequences. This project focuses on identifying anomalous timing relationships in host-based logs. In this presentation, we describe current efforts to augment existing analytic pipelines with timing information in process chains and more general event sequences. We describe neural-network-based analytics using a LSTM (Long Short Term Memory) architecture, as well as a novel hierarchical Bayesian analysis for handling chains of variable length when the available process chain data is sparse. Our analysis focuses on the open-source Tracer Fire datasets, datasets produced during exercises in which known indicators of compromise are generated. 

Keywords

Cybersecurity

Bayesian hierarchical models

Sequential data

Categorical data

Process chain analysis 

View Abstract 2130

First Author

Alexander Foss, Sandia National Laboratories

Presenting Author

Alexander Foss, Sandia National Laboratories

Standardized Bayesian Accuracy for Remaining-useful-life on Censored Data

Accurate methods for predicting component failures and reliability degradation are critical to the sustainment of weapon systems and national security. Bayesian reliability models developed leveraging censored and uncensored data. However, there is no standard arising from the literature for determining accuracy of such Bayesian models. In this work, we offer a solution. Determining the efficacy of these reliability models relies on the accuracy of the produced time-to-failure distributions. To derive the posterior predictive distributions, samples are obtained by simulating data following the observed data's format, including both uncensored and censored observations. Using calculated posterior parameter draws obtained through Markov Chain Monte Carlo (MCMC) simulations, sample data is generated within the same timeline, resulting in a posterior predictive distribution comparable to the observed data, providing precise method to obtain accuracy in Bayesian statistical models. We discuss how Northrop Grumman Corporation is utilizing this measurement of accuracy on weapon systems to inform future sustainment needs and current readiness. 

Keywords

Bayesian Statistics

Model Accuracy

Data Censoring

Posterior Predictive Checks 

View Abstract 3240

Co-Author

Ragan Suarez

First Author

Spencer Ebert

Presenting Author

Spencer Ebert

WITHDRAWN Evaluating the performance of RFID fixed readers for locating nuclear material storage containers

We explore the use of commercial off-the-shelf Radio Frequency Identification (RFID) technology for nuclear material inventory management. Specifically, we evaluate whether a network of fixed RFID readers and antennas can reliably identify and locate individual nuclear material storage containers (e.g., SAVY and slip top containers) within a complex environment of tagged containers placed on shelves, carts, and in gloveboxes. Such a network of sensors could be used to automatically recognize deviations in operations or to speed up and assist a human accounting of nuclear materials. Data are collected from a set of designed experiments using a network of fixed RFID readers and RF antennas installed in a mock nuclear material storage environment of multiple single- and double-sided double-station gloveboxes, where we vary properties of the sensor network, such as placement and number of antennas, RFID reader settings, and the spatial configuration of tagged containers. We present results on the performance of probabilistic machine learning models trained to identify the presence and detect the location of containers within this environment. 

Keywords

RFID

Nuclear material control and accountability

Machine learning

Design and analysis of experiments 

Abstracts


Co-Author(s)

Justin Strait, Los Alamos National Laboratory
Brian Weaver

First Author

Mary Dorn, Los Alamos National Laboratory