Rahul Upadhyay
ID: CCE932867
Rahul Upadhyay
ID: CCE932867
- Data engineer, Lagozon technology
- Uttar Pradesh, India
Rahul Upadhyay
ID: CCE932867
- Data engineer, Lagozon technology

About Rahul Upadhyay
Experienced Data Engineer with over 6 years specializing in real-time data processing. Skilled in fetching and processing streaming data from Kafka using Spark Streaming within Databricks environments. Adept at designing scalable data pipelines and optimizing data workflows to support timely, reliable analytics and decision-making. Committed to leveraging advanced technologies to build efficient, robust data solutions that meet complex business needs.
Employment History
Lagozon technology Invite company
7th Apr 2023 to Present
Data engineer
7th Apr 2023 to Present
Salary Package
Salary Hidden
Roles & Responsibility
Role involves developing and maintaining real-time data pipelines using Apache Spark Streaming to ingest and process data from Apache Kafka. Responsibilities include applying transformation and aggregation functions to streaming data for analytics and business insights. Expertise required in optimizing Spark jobs for performance and scalability within Databricks environment. Additionally, implement robust data writing strategies to efficiently load aggregated results into PostgreSQL databases ensuring data integrity and consistency. Collaborate with cross-functional teams to support data-driven decision making and maintain system reliability.
Skills
- Snowflake
- SQL Azure
- Python ,
- Databricks
- AWS Cloud Computing & Devops
- Azure Data Factory
Verification Pending
Education
Btech in electronic communication
Rajiv Gandhi
1st May 2012 to 30th Jun 2015 (Expected)
Expertise
SQL Azure
4/5
pyspark
4/5
Databricks
4/5
Python
3/5
Lingo
English
Verbal
4/5
Written
5/5
