SUMMARY

The Senior Data Engineer is responsible for designing, building, and maintaining robust data pipelines using Python, SQL Server (T-SQL, Stored Procedures), and integrating various APIs within a modern Azure infrastructure. The role supports critical data operations in regulated environments.

JOB RESPONSIBILITIES

• Design and implement scalable ETL pipelines with Python and SQL Server for data ingestion, transformation, and integration from diverse sources, including RESTful APIs.
• Develop and optimize T-SQL stored procedures to fulfill business logic and reporting needs.
• Architect secure, efficient data workflows leveraging Azure Data Factory, Azure SQL, and Azure Functions.
• Ensure data quality, lineage, and compliance with regulations such as HIPAA and SOC2.
• Collaborate with software engineering teams to align data structures and APIs with application requirements, including those based on .NET and C++ environments.
• Work cross-functionally with data analysts, security engineers, and DevOps to support comprehensive data needs.
• Monitor and troubleshoot pipeline failures and data discrepancies.
• Lead data governance initiatives, best practices, and performance tuning efforts.
• Mentor junior engineers and participate in peer code reviews.

QUALIFICATIONS

• 5+ years of experience in Data Engineering or a similar role.
• Strong proficiency in SQL Server, including stored procedures and performance tuning.
• Advanced skills in Python for ETL development and automation.
• Hands-on experience with Azure data services such as Data Factory, Blob Storage, and Azure SQL.
• Experience integrating data from RESTful APIs and third-party sources.
• Knowledge of data privacy and compliance standards, especially HIPAA and
• SOC2, relevant to financial and healthcare sectors.
• Strong problem-solving abilities and capacity to work independently in a fast paced, regulated environment.
• Excellent written and verbal communication skills.
• Experience with additional Azure services like Azure Synapse, Azure DevOps, and Azure Functions.
• Understanding of data modeling and warehousing concepts.
• Exposure to cybersecurity data, SIEM tools, or SOC operations is advantageous.
• Background in MSP/MSSP environments or consulting.
• Familiarity with Power BI or other data visualization tools.
• Knowledge of .NET Framework and C#-based APIs, particularly in data consumption contexts.

JOB REQUIREMENTS

• Should be willing to accept a long-term work-from-home arrangement.

• Should be amenable to a permanent night shift schedule.