Find Jobs
Hire Freelancers

bigdata project

$10-30 USD

已取消
已发布超过 7 年前

$10-30 USD

货到付款
In this project, you will be asked to work on the ETL concept. ETL stands for Extract, Transform, and Load process. The extract phase may refer to the process of obtaining and mine the data required for the analysis project. This may include some cleaning and combining of the data. The transform phase is the process to make the acquired data comply with the format you are planning to use in the future steps. The load phase is about shipping the data into the enterprise systems (e.g. the database). In this project our focus is on the first two stages, namely Extract and Transform. We may see the last stage in a future assignment. You will be given a group of files. These files represent the customer reviews of a group of products. These products are: Canon G3 camera, Dvd player, Jukebox, Nikon Coolpix, and Nokia 6610. These review files are semi-structured in a format very specific to the website that generated this data. Usually this format limits the gains that can be attained from this data. To overcome this, you need to change this format into a popular format which is the JSON format. Please see below: 1. General Information: Data Description a. You will be given a folder that has the review data files. Each file is fully about one product. b. Each file is semi-structured in the following manner: i. The top of the file has a block of information related to the source of the data. This part is important but it is not of an interest for us in the moment. 1. This block will be ignored. ii. Any review starts with [t] followed by the review title 1. Ex: [t]great camera iii. Any positive aspects will start with [+n], where n can be 1, 2, 3….or any number of points representing a good score. Followed by ## and then the text of the review. 1. Ex: [+2]##i have only had this camera for one full day and i have to say that it is wonderful . iv. Any negative aspects will start with [-m], where m can be 1, 2, 3….or any number of points representing a bad score. Followed by ## and then the text of the review. 1. Ex: [-1]##* main dial is not backlit . 2. First part: Extract, Clean, and Combine a. In this part you will extract the data from the files given in point 1 above. b. In the extract process, you have to identify the product that is being reviewed. c. After identifying the product, each review has to be identified by the title. d. In a review you will identify the positive and negative aspects. To distinguish both, you can look and point 1 above. i. Extract the positive aspects of a review ii. Extract the negative aspects of a review. e. You should know that the text of any review, title and content, will not be clean. The cleaning should be done in the following manner: i. The text should not contain any special characters (!,@,#,$,%,^,&,”,’,<,>,/,? and *) ii. Also speech punctuation symbols have to be removed (, ; : \t) iii. The reviews are written in English. If for any reason a review has a language other than English, you can ignore such a review and do not include it in the output. f. After cleaning all the positive aspects, these will be combined in one single text block of positive criticism. g. The same will be done to the negative aspects. 3. Second part: Data Transformation a. After extracting, cleaning, and combining every review, the data will be put into the outpu
项目 ID: 11657395

关于此项目

5提案
远程项目
活跃7 年前

想赚点钱吗?

在Freelancer上竞价的好处

设定您的预算和时间范围
为您的工作获得报酬
简要概述您的提案
免费注册和竞标工作
5 freelancers are bidding on average $31 USD for this job
用户头像
Hi, I'm Abhijit Mondal from Bangladesh and my background is in Computer Science and Engineering at Bangladesh University of Engineering and Technology. I am an expert Python, Java and Android developer and I have 5 years of experience coding with these languages. I have read the job description you provided above. I am very much familiar and interested to develop your required software with Python/Java. If I get the proper job description, I will start from right then working on it as I am completely free to work in next few weeks. I think it will be nice working for you.
$61 USD 在1天之内
5.0 (5条评论)
3.0
3.0
用户头像
We can use Apache Spark to do this. It has by default inbuilt libraries to read the unstructured data and convert into JSON. Further preprocessing can be done. I have worked with spark and NoSQL databases as well like Cassandra and HBase.
$40 USD 在3天之内
4.8 (2条评论)
2.2
2.2
用户头像
experienced in handling big data projects experienced in python used pandas module in python for handling big data projects
$15 USD 在3天之内
0.0 (0条评论)
0.0
0.0

关于客户

UNITED STATES的国旗
United States
0.0
0
会员自9月 25, 2016起

客户认证

谢谢!我们已通过电子邮件向您发送了索取免费积分的链接。
发送电子邮件时出现问题。请再试一次。
已注册用户 发布工作总数
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
加载预览
授予地理位置权限。
您的登录会话已过期而且您已经登出,请再次登录。