Data Engineer ā Generative AI
About the Role
We are looking for a talented Data Engineer to join our Data team. You will collaborate with research scientists and machine learning engineers to enhance and scale the efficiency of our models. Your primary role will be to build and maintain the data infrastructure that supports the training of all Stability AI models.
š Location: Germany (Remote)
Responsibilities
- Preprocess and normalize data for ingestion into ML training pipelines, ensuring high data quality.
- Design, implement, and maintain scalable data infrastructure for generative AI.
- Develop tools to efficiently search and serve data at scale.
- Collaborate with Research teams to understand and fulfill their data requirements.
- Develop and manage data pipelines to support ML teams.
- Ensure data quality and integrity across various databases and storage systems.
- Manage large-scale unstructured data, including image, text, audio, video, and 3D datasets.
Qualifications
- Experience with large-scale distributed workloads and ML data pipelines.
- Proficiency in Python for data processing and software development.
- Experience with cloud storage & file systems (AWS S3 preferred).
- Familiarity with large-scale data processing and handling unstructured data.
- Expertise in databases, data lakes, and warehouses (e.g., Redshift, BigQuery, Snowflake).
- Experience in machine learning projects, ideally with deep learning/computer vision knowledge.
- Strong teamwork and communication skills for a distributed international team.
- Attention to detail with the ability to document processes and solutions effectively.
Equal Employment Opportunity
We are an equal opportunity employer and do not discriminate based on race, religion, national origin, gender, sexual orientation, age, veteran status, disability, or other legally protected statuses.
