职位描述
Role responsibilities:
• Design data model and build the data foundation by implementing data pipelines in data lake/warehouse, in collaboration with product owners/BSA, data analysts, and business partners
• Contribute to overall architecture, frameworks and patterns for processing large data scale of data
• Build utilities, user defined functions, libraries, and frameworks to better enable data flow patterns
• Profile and analyze data for the purpose of designing scalable solutions
• Define and apply appropriate data acquisition and consumption strategies for given technical scenarios
• Design and implement distributed data processing pipelines using tools and languages prevalent in the big data ecosystem
• Implement complex automated routines using workflow orchestration tools
• Work with architecture, engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and adhered to
• Anticipate, identify and solve issues concerning data management to improve data quality
• Build and incorporate automated unit tests and participate in integration testing efforts
• Utilize and advance continuous integration and deployment frameworks
• Troubleshoot data issues and perform root cause analysis
• Work across teams to resolve operational & performance issues
The following qualifications and technical skills will position you well for this role:
• MS/BS in Computer Science, or related technical discipline
• 5-8 years big data experience on the engineering area like data warehouse, data lake, with good data abstraction sense.
• Experience on data modeling from conceptual to physical
• Strong programming experience, Python is preferred
• Extensive experience with Hadoop and related processing frameworks such as Spark, Hive, etc.
• Experience with RDBMS systems, SQL and SQL Analytical functions
• Experience with workflow orchestration tools like Apache Airflow
• Experience with source code control tools like Github
• Experience with performance and scalability tuning
• Ability to influence and communicate effectively, both verbally and written, with team members and business stakeholders
• Interest in and ability to quickly pick up new languages, technologies, and frameworks
• Experience in Agile/Scrum application development
The following skills and experience are also relevant to our overall environment, and nice to have:
• Experience with Java/Scala
• Experience with Kubernetes, Docker
• Experience working in a public cloud environment, particularly AWS
• Experience with cloud warehouse tools like Snowflake
• Experience with messaging/streaming/complex event processing tooling and frameworks such as Kinesis, Kafka, Spark Streaming, Flink, Nifi, etc.
• Experience working with NoSQL data stores such as HBase, DynamoDB, etc.
• Experience building RESTful API’s to enable data consumption
• Experience with build tools such as Terraform or CloudFormation and automation tools such as Jenkins or Circle CI
• Experience with practices like Continuous Development, Continuous Integration and Automated Testing
These are the characteristics that we strive for in our own work. We would love to hear from candidates who embody the same:
• Desire to work collaboratively with your teammates to come up with the best solution to a problem
• Demonstrated experience and ability to deliver results on multiple projects in a fast-paced, agile environment
• Excellent problem-solving and interpersonal communication skills
• Strong desire to learn and share knowledge with others
招聘负责人
HR
在线沟通
**********
点击查看完整电话
工作地点:
上海-杨浦区
尚浦中心7号楼上海杨浦区江湾城路99号尚浦中心李娜楼
投递简历
温馨提示: 用人单位招聘人才,以任何名义收取费用(如体检费、服装费等)都属于违法,请应聘者提高警惕!
单位其他职位
英语
杭州-西湖区
本科
3-5年
HR
英语
上海-杨浦区
本科
5-10年
HR
英语
合肥-蜀山区
本科
经验不限
HR
英语
杭州-滨江区
本科
经验不限
HR
英语
杭州-滨江区
本科
1-3年
HR
英语
合肥-蜀山区
本科
3-5年
HR
英语
杭州-西湖区
本科
经验不限
HR
英语
合肥-蜀山区
本科
3-5年
HR
英语
合肥-蜀山区
本科
3-5年
HR
英语
杭州-滨江区
本科
1-3年
HR
英语
杭州-西湖区
本科
3-5年
HR
英语
合肥-蜀山区
本科
经验不限
HR