我们目前有一个视频类APP的项目需要招募开发人员多名，要求： 1. 视频系统架构师，要求具备大型分布式视频类系统的架构经验，理解单点系统功能的架构，精通Linux/Unix服务器脚本程序 2. 后台工程师，要求具备FFMPEG二次开发经验，熟悉RTSP协议，熟悉C语言，具备大型项目的工作经验。 3. 前段iOS，Android开发工程师，具备摄像头视频音频采集程序的编写经验。 报酬：优厚，根据每人的能力经验不同而不同，高于市场水平。 工作地点：美国洛杉矶海滨城市Santa Monica，要求必须驻场开发。 周期：第一个项目周期为期3个月，二期视情况而定 视个人情况不同，我们公司可以协助办理工作签证H1B
SQL注入/代码审计/数据库权限/渗透测试技术 接渗透单子高手，联系电子邮箱：aqcp#（#改成@）语言要求：中文 熟练掌握各种内网渗透测试工具与相关知识，精通常见安全攻防技术：sql注入、xss、文件上传、文件包含、命令执行等漏洞 熟悉linux、unix、Windows、oracle、J2EE等各种环境下WEB的安全配置与安全检查及WEB漏洞防范； 熟悉各种脚本语言(asp,php,jsp,java,net、perl、python等)具备独立挖掘web安全漏洞与入侵手段 具备一定的网络安全知识，对网络安全结构、系统漏洞、入侵检测、病毒防护等有深入理论基础和实践经验； 熟悉渗透测试黑客攻防的步骤、方法、流程、熟练掌握各种渗透测试攻防工具有实际渗透经验者优先； 求大虾接单，脱裤，要求，电话，手机，姓名，邮箱，其他的一概不要。每笔单1000美金起步，根据站的难度而定价格,具体可以详谈，寻找长期合作. 态度认真、中介勿扰,请技术实力说话。
...integration and ETL processes Familiarity with MongoDB and data storage + M365 Integration Strong analytical and problem-solving skills Excellent communication and documentation skills Task Breakdown: Develop a fuzzy matching module to compare data across 15-20 points of data Implement additional matching algorithms (e.g., exact matching, regex pattern matching) to handle various data formats Develop 15*3 ETL processes to handle different data formats and request codes Integrate the developed modules with our Azure/M365 environment Configure data storage, authentication, and authorization Perform thorough testing and debugging of the developed modules and ETL processes Deliverables: Fuzzy matching module Additional mat...
I need a Python developer who can handle automation tasks related to ETL pipeline, data extraction from Snowflake, and a dbt environment. Key Requirements: - You should be proficient in Python programming. - Be familiar with the Window Task Scheduler. - Have experience with ETL pipeline development. - Know how to extract data from Snowflake and a dbt environment. The Purpose: - The ETL pipeline is crucial for handling data analysis, reporting, and data integration tasks. - The data size to be processed in the ETL pipeline is medium, up to 10 GB. Ideal Candidate: - You should have prior experience in handling similar tasks. - A strong understanding of data handling and automation through Python and ETL tools. - Proven expertise in data extraction from...
This is a task. Analytical details will be given in docx in chat. We need a freelancer with proven experience and good expertise in systems programming using C or C++, proficiency in UNIX/Linux system APIs for process and IPC management, and experience in concurrency control and robust testing. Familiarity with shell scripting and version control using Git is also essential. 1. Implement jobCommander and jobExecutorServer for user command interface and job management. 2. Allow dynamic setting of concurrency levels for job execution. 3. Use named pipes for communication between jobCommander and jobExecutorServer. 4. Manage jobs with commands like issueJob, setConcurrency, stop, poll, exit, including unique job IDs. 5. Handle job completion signals with `SIGCHLD`. 6. Provide a READM...
...ensuring accuracy and consistency. - Data Modeling: The ideal candidate should have a deep understanding of how to organize data in a way that supports business needs and decision-making. - ETL Processes: The selected freelancer will be responsible for the extraction, transformation, and loading of data into our warehouse. We are working with a medium-sized dataset (1-10 terabytes), so we need someone who is not only experienced in data warehousing but also with datasets of this size. Ideal Skills and Experience: - Proven track record in data warehousing, with a focus on data integration, data modeling, and ETL processes - Experience working with medium-sized datasets (1-10 terabytes) - Excellent analytical and problem-solving skills - Strong communication skills to eff...
I'm in need of a data processing expert who can assist me in developing an architecture to analyze knowledge graphs using ETL and machine learning. Key Tasks: - Leveraging Data Processing Expertise: I need someone with a strong background in data processing, particularly in relation to graph technologies. Your role will be to harness this knowledge to create an efficient and effective architecture for analyzing knowledge graphs. Ideal Skills and Experience: - Strong Data Processing Background: A deep understanding of data processing is crucial for this project, particularly in the context of graph technologies. Experience in working with large datasets and complex data structures is highly desirable. - Familiarity with Graph Technologies: Knowledge of graph databases and asso...
I'm in need of a data processing expert who can assist me in developing an architecture to analyze knowledge graphs using ETL and machine learning. Key Tasks: - Leveraging Data Processing Expertise: I need someone with a strong background in data processing, particularly in relation to graph technologies. Your role will be to harness this knowledge to create an efficient and effective architecture for analyzing knowledge graphs. Ideal Skills and Experience: - Strong Data Processing Background: A deep understanding of data processing is crucial for this project, particularly in the context of graph technologies. Experience in working with large datasets and complex data structures is highly desirable. - Familiarity with Graph Technologies: Knowledge of graph databases and asso...
...to develop a multithreaded program capable of sorting a list of numbers using different sorting algorithms. The program will be implemented in Java and will leverage the multithreading capabilities of the Java programming language. The project will specifically target a Linux environment, providing students with practical experience in developing and running multithreaded Java applications in a Unix-like operating system. Implementation Develop the project using Java programming language. Utilize Java's built-in threading support (e.g., Thread class, Runnable interface) for multithreading. Implement the chosen sorting algorithms as separate classes or methods that can be executed concurrently by multiple threads. Use Java's threading constructs (e.g., Thread, Runnabl...
I'm looking for a skilled data engineer to help me with the OMOP ETL process. Your main responsibilities will be: - Extracting and transforming data: You'll be working with me to extract data from a data warehouse and transform it into the OMOP CDM. - Loading data into a database: After transforming the data, it will need to be loaded into a PostgreSQL database. Experience with ETL processes and PostgreSQL is critical. You should also be familiar with data validation checks, as we need basic data type and range checks applied during this project. You must have experience loading data into the OMOP CDM.
This is a task. Analytical details will be given in docx in chat. We need a freelancer with proven experience and good expertise in systems programming using C or C++, proficiency in UNIX/Linux system APIs for process and IPC management, and experience in concurrency control and robust testing. Familiarity with shell scripting and version control using Git is also essential. 1. Implement jobCommander and jobExecutorServer for user command interface and job management. 2. Allow dynamic setting of concurrency levels for job execution. 3. Use named pipes for communication between jobCommander and jobExecutorServer. 4. Manage jobs with commands like issueJob, setConcurrency, stop, poll, exit, including unique job IDs. 5. Handle job completion signals with `SIGCHLD`. 6. Provide a READM...
I'm looking for an AWS Glue expert to handle my large MySQL database, over 10GB, and write it to S3. You will need to have: - A strong understanding and experience in handling large databases. - Proficiency in using AWS Glue for data ETL processes. - Knowledge in determining the best strategy to quickly load large amounts of data. - Experience working with Parquet output format. The database gets updated daily, so a fast and efficient process is required to ensure optimal performance. Please detail your experience with similar projects when making your bid.
I'm seeking a proficient Power BI expert who can help me create ...BigCommerce, Google Adwords, and Google Sheets - Metrics to Track: Sales revenue, Website traffic, Customer acquisition cost, Inventory levels, and Gross Margin - Update Frequency: Daily Ideal Skills and Experience: - Proficient in Power BI; capable of integrating various data sources seamlessly - Experience in creating dashboards and reports for Sales, Marketing, and Inventory data - Strong knowledge of ETL processes and data visualization techniques - Ability to automate data refresh on a daily basis Your deliverable will be a fully functional and automated Power BI dashboard that visualizes the mentioned metrics accurately and succinctly, providing me with real-time insights into the health and performance ...
As the professional handling this project, you'll engage with big data exceeding 10GB. Proficiency in Python, Java, and Pyspark are vital for success as we demand expertise in: - Data ingestion and extraction: The role involves managing complex datasets and running ETL operations. - Data transformation and cleaning: You'll also need to audit the data for quality and cleanse it for accuracy, ensuring integrity throughout the system. - Handling Streaming pipelines and Delta Live Tables: Mastery of these could be game-changing in our pipelines, facilitating the real-time analysis of data.
I am seeking a skilled data engineering trainer, speed in using Hadoop, Apache Spark, and SQL is paramount. Your expertise will guide me through nuanced uses of these technologies, with a particular focus on data migration. Key Requirements: - Proficiency in Hadoop, Apache Spark, and SQL - More than 10 hours availability weekly - Proven experience in real-world data migration projects Ideal candidates should have a flair for explaining complex concepts in simple language. This engagement will focus on moving data from diverse sources into a data warehouse, thereby making it readily available for business intelligence functions.
As a client, I'm seeking a Python developer who that can help me trasforming data. The primary source of the data will be company-related information. Ideal Skills and Experience: - Proficiency in Python - Extensive experience in data analysis, particularly with telecomdata - Familiarity with APIs for data collection - Knowledge of telecom - Ability to understand Python appl...experience in data analysis, particularly with telecomdata - Familiarity with APIs for data collection - Knowledge of telecom - Ability to understand Python application and python scripts -Know about Airflow , Spark, Kafka, Transaction Mysql Key Requirements: - Develop a Python program that can efficiently trasform non useful to useful data. - Strong understanding of telecom data -Spark sessions, ETL...
...and analysis, within a short timeframe. - Entity-Relationship Modeling: Develop and finalize a comprehensive ER diagram that illustrates the database structure and supports urgent marketing analytics requirements. - Efficient Data Export and ETL Processes: Set up and manage the rapid export of data from various platforms using APIs. Develop and implement ETL (Extract, Transform, Load) processes to clean, transform, and load data into the database swiftly. - Documentation: Produce concise and clear documentation on the data architecture, ER diagrams, and ETL processes to support ongoing maintenance and scalability. - Collaborative and Rapid Execution: Work closely with the project manager and other team members to ensure alignment of the database design with business...
...reports Experience with Automated Testing Five (5) • years of Testing experience in a Software Development Lifecycle environment Five (5) • SQL and PLSQL Programming in SQL Server ETL and Data warehouse process Informatica 10 • 20 Informatica Data Quality Profiles , Score Cards SQL Server RDBMS SQL Server Analytics Services SSAS SQL Server Reporting Services SSRS Tableau Reporting BMC Control M Scheduler Tool Data Analysis Data Profiling Application Life Cycle Management Complete Testing process SDLC, Agile Process, SAFE and Kanban methodologies, TFS Tool Good Communication Skills and Quick Learner Selenium Automation Unix Finance Domain Knowledge Automation Scripting using excel Macros Other Functions: • The Quality Engineer IV participates in the deve...
As an ETL Expert Data Engineer with Airflow expertise, I require a professional who can take on a sophisticated data integration task. Key Responsibilities: - You will be responsible for data integration. - The data will be pulled from various APIs. - The processing logic required is advanced, aimed at facilitating analytics and machine learning. You should possess: - Extensive ETL experience. - Deep knowledge of Apache Airflow. - Proven track record in handling complex data integration tasks. - Capability in building data pipelines for advanced analytics. - Familiarity with APIs, ensuring efficient data extraction. - Strong problem-solving skills and attention to detail. I am looking for someone who can take this project and deliver high-quality results promptly.
...in data migration across CRP, SIT, UAT, and GoLive phases in Oracle Fusion ERP implementation, utilizing ETL tools. * Experience in SQL Server, T-SQL, and SSIS Job Responsibilities: Data Extraction: * Extract data from various source systems, including flat files, spreadsheets, legacy databases, and external APIs. * Develop and optimize SQL queries and scripts for efficient data retrieval. Data Transformation: * Perform data cleansing, validation, and transformation to ensure data quality. * Convert data from source formats to Oracle-compatible formats. * Apply business rules and data mapping for aligning source data with target data structures. ETL Development: * Develop and maintain ETL (Extract, Transform, Load) processes using standard tools. * Handle data integ...
...instant messaging for convenience. - help in exam preparation Here is a list of all the topics covered in this course Building and Introduction - Basics of .o, .a, .so files, dynamic/static linking, and makefiles. Accounts and Permissions - POSIX permissions, user IDs, and sudo functionalities. Privilege and Exceptions - Kernel mode, system calls, and exceptions. Signals and Processes - Handling Unix signals, processes, and threads management (fork, exec). Virtual Memory - Concepts, page tables, and memory management techniques. Caches - Understanding caching, memory hierarchy, and cache-related strategies. Synchronization and Deadlocks - Locks, mutexes, barriers, and deadlock prevention. Networking - IP, TCP, DNS, and secure channel foundations. Secure Channels and Cryptography...
...automation tools and frameworks (CI/CD pipelines). - Conduct systems tests for security, performance, and availability. - Develop and maintain documentation and systems diagrams. **Required Skills and Qualifications:** - Proven experience as a DevOps Engineer or similar software engineering role. - Proficient with AWS cloud services. - Expertise in Docker and Kubernetes. - Strong background in Linux/Unix Administration. - Experience with CI/CD systems. - Proficiency in version control tools like GIT. - Strong problem-solving skills and ability to manage complex and urgent issues. - Excellent communication skills and ability to work independently on projects. **Preferred Qualifications:** - Certifications in AWS, Kubernetes, or other DevOps tools are highly desirable. - Prior expe...
Job Title: Senior Master Data Management (MDM) Developer Monday to Friday Remote via Zoom 2-3 hours/day IST Night (7pm to 12amIST) Salary: 30,000 INR per mont...Monday to Friday Remote via Zoom 2-3 hours/day IST Night (7pm to 12amIST) Salary: 30,000 INR per month We are seeking a Senior Master Data Management (MDM) Developer with a focus on healthcare. Strong SQL skills for data manipulation and querying. Experience with ETL development and integration. Familiarity with Azure Data Factory and cloud/on-prem migrations is advantageous. One needs to be available in the mentioned timings and willing to work by screen share and remote control via Zoom. Skills Required: Master Data Management (MDM) ETL development and integration Strong SQL skills SSIS Azure Data Factory Cloud/o...
I am urgently in need of an experienced C programmer with solid understanding of kernel functionalities and operating systems. Since the undertaking is about adding new features to an existing operating system, the freelancer woul...implement new features based on undefined needs - this calls for a degree of creativity and problem-solving skills as the specifics of the new features were left unstated. - Thoroughly test these features to ensure seamless integration and perfect functionality Please note: The project needs to be completed in 48 hours, hence great time management skill is required. Proven experience in C Programming, unix, system analysis, software development and testing is highly desirable. Note I only want the program written in C, no other variant of the language i...
I am encountering an issue with sending realtime data to Facebook META CAPI. Currently, I am using Zapier to send data, but it's appearing in the event manager in an hourly manner rather than in realtime. Despite following the META guide meticulously and sending data via API in GMT Unix timestamp format, the event manager dataset still reflects hourly intervals. I require someone proficient in resolving this issue and ensuring that data transmission shifts from hourly to realtime. It's important to note that this is not a straightforward IT project involving setting up websites or landing pages. This is a theoretical problem that demands a nuanced understanding and expertise in data transmission dynamics. I specifically urge individuals solely focused on bringing in busin...
As a client, I am in search of a skilled Python developer with experience in XML parsing and handling database operations. I require a Python utility program to parse Talend jobs and determine source and target column details. Key Requirements: * Parse XML files - The Talend mapping file will be supplied in XML format. We're in need of someone fluent with Python's X...or potentially other formats. It should be flexibly designed as we will use this data to load into a database. * Database expertise - An understanding of database interactions and structure are vital. This project will require both a keen eye for detail and the ability to create efficient, effective code. The utility must be well-documented and easy to maintain. Experience with Talend or other ETL tools wo...
I'm constructing a robust real estate scoring model, and need a skillful database engineer proficient in Google Big Query. The project involves: 1. ETL Development: Your main task is to develop an ETL process. You should have experience managing CSV files and integrating APIs. 2. Data Management: The data to handle includes property addresses, sale price history, property characteristics, and people details related to properties. The expert should know how to organize and manage this huge amount of data effectively. 3. Real-Time Updates: Due to the volatile nature of the real estate market, the ability to integrate real-time updates is an asset. An expert in Google Big Query, effective in handling CSV files, and capable of seamless API integration would make the best f...
I'm in search of a Azure Data Factory expert who is well-versed in Delta tables, Parquet, and Dedicated SQL pool. As per the requirement, I have all the data and specifications ready. The successful freelancer will need to be familiar with advanced transform...in search of a Azure Data Factory expert who is well-versed in Delta tables, Parquet, and Dedicated SQL pool. As per the requirement, I have all the data and specifications ready. The successful freelancer will need to be familiar with advanced transformations as the ETL complexity level is high. It's a plus if you have prior and proven experience in handling such projects. Key Skills Required: - Expertise in Azure Data Factory - Deep knowledge of Delta tables, Parquet and Dedicated SQL pool - Familiarity with adv...
Job Description: Develop & support Power BI artefacts and participate in the analytics product development Participate in the analysis of requirements to understand the business and system needs regarding user interfaces, reporting enhancements, new functionality, and back-end ETL operations, should be able to provide estimates against requirements. Code and modify reports using DAX & MDX programming languages. Document all code using standard forms and procedures, adhering to all naming conventions Participate in unit, system, and user acceptance testing Adopt new and emerging technologies to provide solutions to client needs. Additional responsibilities will be team performance management, timesheet reviews, ensuring project delivery on time. Required Primary Skills: ...
"Airflow pipeline I require a professional with experience in Airflow and dbt for my project. You will be handling an ETL/ELT integration specifically for data migration. This migration task involves moving specific tables from one MySQL database to another one. I'm trying to migrate data from mysql to mysql. We need to transfer incremental data on an hourly basis. We are looking to use open source ETL or ELT. In addition to airflow, you can use other open source ETL tools.".
...to be connected to Azure; this is contract management system. 2. One more company recently which was acquired has financial data on-prem on different tenant we need to bring in this data to the Azure cloud. Note : Business central is for the financial data and GL entry ,Dynamics 365 is for the sales and invoicing ,Hubspot for the sales lead, Service Now is the ticketing system , Workato is the ETL tool which connects all the system,Kimble one is the project management platform. There is one billing system acquire going on we might need to bring in the data from this source also. Data Handling: Perform necessary data normalization or transformation before storing the data in Azure SQL Database. The ideal candidate for this role will have past experience with Azure SQL Database...
...with experience in Airflow and dbt for my project. You will be handling an ETL/ELT integration specifically for data migration. This migration task involves moving specific tables from one MySQL database to another one. Your key responsibilities for this project will include: - Identifying the specific tables needed for migration, as indicated by me. - Developing and implementing an efficient migration process using Airflow and dbt. - Ensuring successful and accurate data migration. Ideal skills: - Incremental data must be transferred every hour. - Transfer speed is important. - Extensive experience in Airflow & dbt. - Proficient in MySQL databases. - Experience in data migration projects. - Thorough knowledge of ETL/ELT processes. - Attention to detail as the task ...
...Timings : 10:00 am - 07:00 pm IST Remote (India) • Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines. -Connect with stakeholders daily to discuss project progress and updates. -Work within an Agile process to deliver projects in a timely and efficient manner. -Have worked extensively on Redshift and understands performance tuning techniques and management of Redshift data workloads. -Design and develop Airflow DAGs to schedule and manage ETL workflows. -Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture. -Monitor and troubleshoot ETL pipelines to ensure smooth operation. -Custom Python functions to handle data quality and validation. ...
I need an ETL developer to streamline and automate an existing manual process. This includes: - Transferring two encrypted *. files from a source server to a local machine. - Decrypting these files into *.csv format. - Running SQL on one file and a PowerShell command on the other file to create a staging table. - Setting up a Microsoft SQL Server database into read-write mode from read-only. - Reverting the database back to read-only upon task completion. - Sending a report to the user upon task completion. Ideal skills for this job include advanced SQL, PowerShell, and ETL development. Familiarity with read/write permissions on Microsoft SQL Server and experience with data transfer and encryption/decryption processes are also highly beneficial.
I am seeking an IICS (Informatica Intelligent Cloud Services) specialist to effectively set up and integrate AWS S3/EC2 services with this platform. Key Requirements: - Set up project infrastructure per given specifications - Ensure successful integration of AWS S3 and EC2 with IICS - Establish robust connections and develop ETL processes - Configure parameter files as part of the setup While not specified, an intermediate or advanced level of IICS proficiency would be ideal for this task. Previous experience with AWS services is a must. I have not stated a timeframe for completion, indicating flexibility; however, I would appreciate if interested freelancers could provide their estimated timelines of delivery in their responses.
I'm seeking an expert in Docker container deployment and website script installations. I've 3 scripts that need to be installed on a Unix/Linux VPS, each in a separate Docker container. Tasks: - Installation of Script A, Script B, and Script C - Setting up these scripts in individual Docker containers - Verify the functionality of installed scripts Ideal skills: - Proficiency in Docker - Familiarity with Unix/Linux VPS - Experience in website script installations Your task would be to ensure that the scripts function correctly when housed within their designated Docker containers.
I am looking for an experienced Senior Data Engineer for in...looking for an experienced Senior Data Engineer for interview training. Your primary responsibility would be data cleaning and preprocessing, design and optimize database and implement ETL processes. Key responsibilities include: - Clean and preprocess data to ensure its quality and efficiency. - Design and optimize databases, aiming for both flexibility and speed. - Implement ETL (Extract, Transform, Load) processes to facilitate the effective and secure moving of data. Skills and Experience: - Proficient in Python, SQL, and Hadoop. - Expertise in handling medium-sized databases (1GB-1TB). - Proven track record in ETL processes handling. Your expertise in these areas will be crucial to the successful com...
I wrote a Java program that I want to sell to users. I want to make sure that no one can reverse-engineer the...that no one can reverse-engineer the byte codes to recover my source code. I am aware of the concept of obfuscation, but I have no experience with specific obfuscators. I'm looking for someone with experience using obfuscators. I want you to: 1) Recommend an obfuscator. 2) Explain why you recommend that obfuscator and no other. 3) Show me how to download and install that obfuscator. 4) Create a UNIX shell script which applies that obfuscator to my Java program. This is a fixed price contract. The total value is $250.00 USD, which will be paid upon successful completion of the steps above. Please do not apply for this project unless you have already done this at lea...
Objective: Upgrade an existing ETL process to enhance data validation, error handling, and user feedback for structured Excel files uploaded to SharePoint 2019, for subsequent transfer to a database. The solution could be also implemented via Azure Functions and Sharepoint online if deemed possible. Existing Setup: A basic ETL process developed with the Luigi library, lacking advanced validation and user feedback mechanisms. Solution uses parsing code (Pandas) for Excel files data parsing that can be utilized for the upgrade also. There are 4 different excel files, so 4 different parsers will be provided. The system should manage four unique Excel file types, each in its own SharePoint folder, requiring separate parsers for their distinct structures. An initial selection...
Looking for an experienced MSBI and powerbi developer. You would be working with me to troubleshoot and optimize an existing MSBI solution and powerbi report. Your tasks include but are not limited to: - Data Modeling - ETL Development - Report Development For this project, your primary tools would be: - SQL Server Integration Services (SSIS) - SQL Server Analysis Services (SSAS) - power bi Your experience and skills in these areas will be critical to the success of our project. Only serious and high-quality professionals need apply. This is a challenging role and previous experience in the given areas is a must. Let's enhance our MSBI solution together and make it the best it can be.
1) Continue on previous projecName existing api to: api/v1/log/post 12 New api: api/v1/log/get-time/<Serial> • For instance, api/v1/log/get-time/12344 • Returns unix time from Redis, 0 if not exist 3) Add customizable numeric return value for API functions • , return OK codes for all calls – api-user must decode different numeric values. 4) Change inserts to use COPY 5) Change to unix time in post message 6) Add redis data cache. • Lookup serial number in redis, if exist then process post-message • Add unix time from the first line as value to the serial key 7) Restructure user/database, something like this (a log_user shall only be allowed to INSERT/COPY into log_data in Postgres:
...manage and organize data within it. - Ability to ensure data security and integrity throughout the extraction and integration process. **Ideal Skills and Experience:** - Proven track record of working with POS systems, especially Clover, for sales data extraction. - Strong background in cloud-computing platforms, specifically Azure. - Comfortable with setting up and managing data pipelines and ETL processes. - Experience in setting up automated workflows between different technological platforms. - Knowledge of data visualization and interpretation, to guide the final format of sales data within Snowflake. This project requires a freelancer who can swiftly understand our data infrastructure and implement a solution that minimizes manual intervention and maximizes data utility f...
...guidance especially in the area of ETL processes. I need to focus on: - This is an interview support role, You are supposed to help in live interviews. • Extraction techniques – The primary data sources of my interest are platforms like Spark, AWS, Azure, GCP, and Hive. I want to understand effective methods for data extraction from these particular sources. Ideal Skills and Experience: - Expertise in ETL tools for data extraction - Hands-on experience with Spark, AWS, Azure, GCP, Hive - Profound knowledge in data engineering - Experience in career coaching or mentoring will be a bonus - SQL -Python This assistance will give me a competitive edge in my upcoming interviews by providing me with practical skills, knowledge, and confidence in addressing ETL...
I'm seeking an experienced Data Engineer with proficiency in SQL and PySpark. Key Responsibilities: - Develop and optimize our ETL processes. - Enhance our data pipeline for smoother operations. The ideal candidate should deliver efficient extraction, transformation, and loading of data, which is critical to our project's success. Skills and Experience: - Proficient in SQL and PySpark - Proven experience in ETL process development - Previous experience in data pipeline optimization Your expertise will significantly improve our data management systems, and your ability to deliver effectively and promptly will be highly appreciated.
More details: Which specific programs do you want to install and run using Cygwin? Program A What functionality are you ex...type of files will you be running through Program A in Cygwin? py, Mol2, dms The programmer should get access to my laptop and manage to install cygwin correctly, because I failed and bashrc file was not in Home directory - actually, I could not even find it anywhere. And, I think that its environment should be at home/usrname where unix normally is. But it was in my windows env under users/me. Meaning I had a GUI (Windows) for my files. Which I should not (Unix). So, overall wrong. So, even if I installed programs I could not run anything. I am not requesting you to run my files. I want you to set the program for me to do my jobs. I hope that is...
Looking for a seasoned AWS expert to assist in optimizing EC2 instances, S3 storage, and Lambda functions for improved performance. The ideal candidate will have: Mainly working on Aws glue jobs , S3,lambda, Crawler, step functions, dynamo DB ,Clouldwatchs, Appflow, parameter store , API Gateways, ETL Jobs,splunk logs CloudFormation - important big bucket,Spunk, Jenkins pipelines* CI/CD * , YAML for creating templates and manifest files * Sql mandatory for date base * using Postman for API calls * Python and spark Musk know
