Reading and Writing Parquet nested datatype file using Pyspark

$2-8 USD / hour

已关闭

已发布

将近 3 年前

$2-8 USD / hour

write a pyspark job which should read a parquet file which has nested datatypes & values( records) and change the one of the column value with xxx and write into a new parquet file. so the actual source file and newly created file should be same with the small change ( changed to xxx for of the row value for one column), the new parquet file should same as the original file (schema, no of records, order of records) with the changed one of the column value 'xxx' Note:- logic should be dynamic ,parquet file schema will not be the same all the time.....our code should read the parquet file schema dynamically and and create the parquet file with changed data ( xxx) ....the rows, schema and columns should be same

PySpark

项目 ID: 31013380

关于此项目

3提案

远程项目

活跃3 年前

想赚点钱吗？

电子邮箱地址

在Freelancer上竞价的好处

设定您的预算和时间范围

为您的工作获得报酬

简要概述您的提案

免费注册和竞标工作

3威客以平均价$7 USD/小时来参与此工作竞价

@MohammedSulhi

I read your project description carefully. I am bidding on your project because I am very much familiar with Python, Pyspark and Parquet. I am an experienced Data Scientist and Machine Learning Engineer. Data Visualization, NLP, Deep learning, Artificial intelligence, machine learning, Data structures, and algorithms are my major fields. I finished specializations on Data Science, Machine learning, Deep neural Network, Convolution NN, Recurrent NN, Tuning Hyper Parameter .this project will well fit for me. I have won the 2nd runners up award in Sri Lanka Biggest Data Science Competition. I am very fluent with python and did a lot of data science and ml project. So I am familiar with these related libraries such as matplotlib, seaborn, pandas, numpy, sikit-learn, Keras, TensorFlow, spark etc . I am an expert in R language and did lot of projects to data visualization , data manipulation and supervised and unsupervised learning.

$5 USD 在40天之内

0.0

(0条评论)

2.0

@jainabhay1796

I will work on this , if I am given opportunity. So basically I would like to discuss few things before I am alloted to this

$8 USD 在40天之内

0.0

(0条评论)

0.0

@jsljaisingh

I do have 6 years of IT experience with Python, pyspark, Spark, SQL, ETL, data analysis and data engineering. I do have exposure to Azure and Aws cloud platforms. Though, i am quite new to this platform but i can assure you i do have a rich experience in robust and scalable data pipelines using pyspark. i have handled static and dynamic schema in feeds. We can have a 20 mins session to understand your needs. Let's connect to discuss more about same.

$7 USD 在20天之内