Data Engineer
$76K-$140K (Glassdoor est.)
Location: San Francisco, CA- United States
Job ID: 20WD39619

Position Overview
Autodesk is seeking a Data Engineer with experience building production data pipelines to join our Data Platforms and Insights team. The Data Platforms and Insights team is chartered with building robust and resilient pipelines that create the master data sets that serve as the building blocks for all data scientists and data analysts throughout the organization. In addition, this team establishes and promotes the data engineering standards and best practices across Autodesk and help to lead the effort to make data an integral part of decision making at Autodesk. The ideal candidate will be responsible for designing, developing, implementing and documenting data solutions for complex ETL data pipelines, and lending technical expertise within the group. The ideal applicant will be able to thrive in a highly collaborative workplace and actively engage in the development process.

Responsibilities
Map and automate data flow from a variety sources including desktop, web, and mobile product feeds and business systems
Develop workflows for data science workloads utilizing orchestration tools (e.g. Airflow) to manage Spark workloads
Deploy data science pipelines to support production data science models (e.g. Sagemaker)
Deploy solutions via automated CI/CD processes to execute data pipelines running in batch
Work with programming languages used for data manipulation, including Scala, PySpark, Spark SQL, R, or Java
Design, develop, execute and document software solutions to address complex data collection, processing, transformation and reporting issues
Collaborate with peer organizations, dev ops, support organizations on technical issues and provide guidance
Interpret and translate business needs to technical requirements
Work with team to troubleshoot code level problems quickly and efficient on as need basis
Develop, refine and educate the data community on coding standards and best practices
Participate in code and document reviews
Minimum Qualifications
Bachelor’s Degree or relevant experience in the field of Computer Science, Mathematics or Statistics
2 to 3 years of experience in big data technologies including Spark, Hadoop, RedShift, Vertica, Snowflake, Hive, Lambda, Glue, S3, Airflow
Proficiency in executing analysis using SQL, Hive, R, Spark Data frames, Scala, and/or Python
Strong SQL skills
Details-oriented with a focus on best practices for design and implementation of scalable data engineering solutions
Fluency with one or more of the scripting languages: Python, Java, Scala, etc.
Working knowledge and experience with basic and complex data structures
Good communication skills and ability to explain complex topics to a non-technical audience
Knowledge and experience with large data sets, event streams and distributed computing (Hive/Hadoop, Spark etc.)
Experience working with Agile Scrum Teams
Experience in Version control management - tracking and coordinating engineering work
Proven experience of finding bugs and vulnerabilities with automated tools
Expertise in data research/analysis with a focus on data quality and consistency
Experience with algorithms, distributed storage & compute to solve complex business problems
Preferred Qualifications
Experience with container systems like Docker and container orchestration like
EC2 Container Service, Kubernetes, Terraform.
An advanced school degree
Autodesk
4.0
Autodesk
