Test Systems Engineer Senior-Level 112-044

Test Systems Engineer (Senior):

Overall Assignment Description:

Senior-level Test Systems Engineers assist in leading Application, System and Integration Testing teams performing test and evaluation across NSG, ASG and Federal Agencies to ensure timely and accurate GEOINT. This task is focused on providing technical advisory support to identify capabilities, tools, technologies, and new agile methods and techniques ready to adapt to current innovations within AI R&D for Test and Evaluation (T&E) Automation and Verification of AI 

models. 

Duties include: 

• Perform appropriate diagnostic testing on AI/ML systems and applications, measuring performance and efficiency on said systems/applications, documentation, and maintenance solutions for systems or subsystems and making recommendations to the government. 

• Create and maintain technical framework to execute specifications to pass system level  integration 

• Prepare and publish presentations and participate in conferences, webinars, and online events

• Design and Develop quantitative, qualitative, or mixed-method tools, techniques, and methodologies to analyze, assess, benchmark, and monitor AI risk and related impacts. 

• Assists the Government in providing requirements engineering, solutions engineering, scheduling, reliability, resiliency, services development, integration, test and evaluation, maintainability and analysis across the NSG, ASG and Federal Agencies. 

• Assist Government in the creation of foundation of AI Assurance capabilities, including thorough formal methods for trusted and trustworthy eXplainable AI (XAI) and /or Independent Verification & Validation methodologies (IV&V) for AI (beyond T&E), T&E/V&V-As-a-Service

• Advises the Government on proposed changes to the solutions designs based on analysis of requirements and new technology towards AI Assurance for best AI Governance. 

• Formulate descriptive, diagnostic, predictive and prescriptive insights/algorithms and translate technical specifications into development code. 

• Conduct statistical analysis and interpreting the results to guide the government’s decisionmaking process.

• Create and lead algorithm governance process that leverage metrics to ensure performance and compliance with the principles established by the DOD and the ODNI  Document procedures for installation and maintenance, complete programming, perform testing and debugging, define and monitor performance metrics. 

• Rapidly iterate AI models and results to refine and validate approach for accuracy and repeatability. 

• Create, Develop and Define business processes and acceptance criteria to support testing and evaluation.

• Author test planning documentation to include test cases and reports and experience running test scenarios within Dev/Test Environment and identify and report potential issues 

• Develop deep learning frameworks and models, GPU acceleration, model optimization, real-time streaming data, and distributed computing.

• Develop test procedures to support the AI model independent verification and validation approach and sync with cross functional groups for alignment.

• Evaluate and test the front to end acceptance criteria of the full AI model architecture to provide feedback to the developer teams and the government 

Skills and Experience: 

Required: 

• Senior level working experience in understanding scripting, statistics, and a have computer vision background .

• 10+ years’ experience with three or more of the following: JMeterand, Jenkins, Docker, Postman, Swagger, Nexus, Apigee or GitHub/GitLab (or equivalent software packages), Java, Bash, Curl,  XML, JSON, SQL, Python, Javascript, or AWS or C2S. 

• Experience in both quantitative and qualitative analysis to support sound metrics to uphold the confidence of the AI model assurance. 

• Senior level working experience with traditional, Agile, and DevOps development practices and associated testing strategies. 

• Experience using Test Driven Development leveraging computer programming languages to include but not limited to; Python, C++, Matlab, R, Java 

• Experience in the field of geospatial intelligence 

• Knowledge of AI Assurance, T&E or V&V-As-a-Service, and XAI. 

• Experience designing quality engineering solutions using Automation, Augmentation and  Artificial Intelligence technologies 

• Experience coding and developing test and evaluations for AI programs 

• Experience in SAFe framework and Model Based Systems Engineering 

Desired: 

• Demonstrated experience in Atlassian products such as Confluence and JIRA 

• Demonstrated experience working with NGA Enterprise solutions to include CI/CD pipeline and DevSecOps 

• Demonstrated experience engineering solutions using Cloud-based technologies. 

• Demonstrated experience with internal and external program transitions and technical capabilities

• Demonstrated experience engineering solutions using structured and unstructured Big Data.

Education and Experience:

• 12 to 18 years' experience.

• Bachelor’s degree or higher in Mathematics, Statistics, Computer Vision, Artificial Intelligence, Machine Learning, or related STEM degree program, or equivalent expert level work experience as a Test Systems Engineer. 

Security Clearance:

• DoD Approved Clearance and Poly