AI Training Automation Engineer - Bash/PowerShell

Role Description
As a remote, hourly paid Bash/PowerShell Engineer, you will review AI-generated responses and generate high-quality automation-focused content. You will evaluate the reasoning quality and step-by-step problem-solving behind scripts and troubleshooting workflows. You will assess solutions for accuracy, clarity, and adherence to the prompt; identify errors in logic, safety, or portability; fact-check technical details; write expert-level explanations and model solutions that demonstrate correct scripting patterns; and rate and compare multiple AI responses based on correctness and reasoning quality. This contract role is with SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate that provides AI training data for many of the world’s largest AI companies and foundation model labs, directly helping to improve the world’s premier AI models.

Your Profile

• Expert-level proficiency in Bash and/or PowerShell scripting.
• 2–3+ years of hands-on experience in at least one of Bash or PowerShell.
• Strong experience writing, debugging, and maintaining shell or object-based automation scripts.
• Ability to evaluate scripts for correctness, readability, safety, performance, and portability across environments.
• Professional experience in software engineering, DevOps, IT automation, or infrastructure-focused roles.
• Minimum Bachelor’s degree in Computer Science or a closely related technical field.
• Significant experience using large language models (LLMs) while coding, troubleshooting, and reviewing scripts.
• Excellent English writing skills with the ability to document and explain complex automation clearly.
• Highly preferred: For Bash, strong comfort with awk, sed, and grep; for PowerShell, experience with PowerShell Core, DSC, and Azure PowerShell modules.
• Preferred: Merged PRs in CLI, DevOps, or infrastructure-related open-source projects; familiarity with CI pipelines, automation workflows, or cloud scripting; previous AI data training or model evaluation work strongly preferred and Minimum C1 English proficiency.

Key Responsibilities

• Develop AI training content by creating detailed prompts across various topics and high-quality responses to guide AI learning, ensuring models reflect a comprehensive understanding of diverse subjects.
• Optimize AI performance by evaluating and ranking AI responses to enhance accuracy, fluency, and contextual relevance.
• Ensure model integrity by testing AI models for potential inaccuracies or biases and validating their reliability across use cases.

Location: Remote - Spain, Brazil, Indonesia, Portugal, India, Bangladesh, Egypt

Skills required for this job:

• Bash
• CI/CD
• Cloud computing
• DevOps
• Infrastructure
• PowerShell
• Software engineering