Automation Engineer

3 weeks ago


San Francisco, California, United States Invisible Technologies Full time

Company Description

Invisible Technologies offers unusual services (a combination of outsourcing and automation) to fast-scaling, innovative companies. Each company understands and uses our services in a variety of ways. It's critical we can communicate to segments of users with visually appealing, and highly communicative assets. You're helping us explain the benefits of our product in a way that words alone never could.

Job Description

Job Responsibilities:

Create crawlers for each source URL by using Python modules [scrapy, selenium, requests, BeautifulSoup and splash]

Create and maintain scrapy pipelines and middlewares to manage the output from the crawlers.

Create crawlers for all types of websites irrespective of the technical roadblocks.

Manage the crawlers to overcome technical challenges like IP ban, geolocation ban, captcha and bot blocking services.

Write SQL queries to manage database operations using Python modules like sqlalchemyDeploy the python scripts / crawlers to Linux based AWS servers

SkillsMust Haves:

Strong hands-on experience in Python programming

Good experience with scraping libraries such as Requests, BeautifulSoup, Selenium and Scrapy.

Proven 3+ experience working on web development frameworks such as Flask/Django/FastAPI/Tornado/PandasKnowledge of building APIs and services using REST.

Experience with any RDBMS and strong SQL knowledge.

Clear with Object-oriented concepts.

Excellent troubleshooting skills.

Proficient understanding of code versioning tools like git.

Basic understanding of front-end technologies, such as JavaScript, HTML5 and CSS3Understanding of fundamental design principles behind a scalable application.

Nice to have:

Familiar with UI frameworks like Angular/ReactJS

Strong unit test and debugging skills.

Strong knowledge of data engineering platforms such as Airflow.

Experience with the Google CloudDockerAgile and Scrum. Even better if have worked in Jira and confluence.

Experience with ETLCeleryExcellent interpersonal skills and ability to to work with a diverse team

Qualifications

Required: Technical Skills

  • Python
  • JavaScript
  • PostgresSQL
  • Web Scraping using Requests, BeautifulSoup, Selenium and Scrapy
  • Frameworks such as Flask/Django/FastAPI/Tornado/Pandas
  • Building APIs and services using REST

Additional Information



  • San Francisco, California, United States ITJobsList Full time

    we are searching for an Automation Engineer to create as well as execute automated and exploratory testing to make sure program quality. Automation Engineer duties include developing as well as using defining debugging and tests remedial actions. The job is able to span through automating Smart Contract testing to our front end portal to producing an end...


  • San Jose, California, United States Zenaide Technologies Inc. Full time

    Minimum 3+ yrs of experience as an automation engineer. QA experience in Network Functions Virtualization VMware vSphere Experience in network testing. Strong experience writing automated tests using Python. Experience writing and following test plans and test cases. Excellent problem-solving skills.


  • San Francisco, California, United States Atechstar Full time

    JOB DESCRIPTIONWhat You'll Do Develop and execute test cases for new product enhancements Identify areas of risk and mitigate those areas to ensure stable releases Ensure quality criteria is met for all areas before product release Other duties as assignedWho You Are Bachelors degree in Computer Science/Information Technology or related field; or software...


  • San Diego, California, United States Eli Lilly and Company Full time

    At Lilly, we unite caring with discovery to make life better for people around the world. We are a global healthcare leader headquartered in Indianapolis, Indiana. Our employees around the world work to discover and bring life-changing medicines to those who need them, improve the understanding and management of disease, and give back to our communities...

  • System Engineer

    3 weeks ago


    San Francisco, California, United States Tata Technologies Full time

    Systems Engineer will be responsible for developing System requirements, Test cases, Test Automation & Power Train systems issue RCA support for an autonomous vehicle.The role requires a highly motivated and hands-on team player with an exceptional attention to detail. The role will suit an Engineer who has the ability to quickly develop skills in new areas...

  • System Engineer

    3 weeks ago


    San Francisco, California, United States DBSI Services Full time

    Benefits:401(k)401(k) matchingBonus based on performanceJob Role: System EngineerLocation: Foster City, CA OR San Francisco, CAThe Systems Engineer will be responsible for developing System requirements, Test cases, Test Automation & Power Train systems issue RCA support for an autonomous vehicle.The role requires a highly motivated and hands-on team player...

  • Founding Engineer

    1 week ago


    San Francisco, California, United States Meticulate Full time

    Meticulate is an AI engine that automates business research & analysis. We build extensive LLM pipelines that automate a huge chunk of the work that millions of investors, consultants, and business decision-makers do day-to-day. We're a seed-stage startup coming out of Y Combinator backed by leading AI investors including Link Ventures, Cervin Ventures, and...

  • Founding Engineer

    2 weeks ago


    San Francisco, California, United States Innercircle Full time

    Innercircle is a Network Relationship Management (NRM) solution that seamlessly integrates Networks, CRM, and Sales Enablement to improve the efficiency of any sales team. Rather than relying on cold emails and cold calls, our customers use InnerCircle to programmatically analyze and leverage their existing connections to generate new business. Our...


  • San Francisco, California, United States Analog Full time

    About Us:Analog is building a true Layer-0 blockchain network that ushers in the omnichain future, secured with the Proof-of-Time consensus mechanism.Backed by world-class investors, our vision is to facilitate the seamless communication of time data (events) between decentralized applications across different chains.We're hiring an experienced Web3 QA...

  • Lead test engineer

    2 weeks ago


    San Francisco, California, United States Atechstar Full time

    Job DescriptionJoin our atechstar family and you'll grow your expertise and become the best version of you. As you embark on a new journey you'll seek challenges with flexibility and patience learning new skills and advancing your career while having the time of your life.The Lead Test Engineer is responsible for leading Test Engineering teams and is...


  • San Francisco, California, United States Ripple Full time

    Developer Operations (DevOps) at Ripple is responsible for communication, collaboration and integration between the Development and Operations teams so infrastructures, tools and processes are streamlined and effective for faster and automated delivery of products.The Staff Software Engineer, DevOps will contribute in discovery, design and implementation of...

  • Backend Engineer

    3 weeks ago


    San Francisco, California, United States Trizone Full time

    Job Title: Backend EngineerLocation: San FranciscoAs a Backend Engineer at Trizone, you will be responsible for designing and implementing the backend infrastructure and systems that power our innovative construction tech solutions. You will work closely with a cross-functional team of software engineers, product managers, and data scientists to build and...


  • San Francisco, California, United States Quid Full time

    Senior DevOps EngineerAs a Senior DevOps engineer for the Quid product you will build software and processes to enable engineers to self-service the operation of NetBase Quid at scale and support 24x7 operations. Our developers are your customers. Your goal is to continuously assess and ease pain points of fellow engineers and of our Cloud infrastructure. We...


  • San Francisco, California, United States Motion Recruitment Full time

    This company develops technology that protects the integrity, confidentiality, and privacy of healthcare data while enabling consumers to comprehend it. They create systems that exhibit an accurate picture of the underlying data, are auditable and automated as far as feasible, and, most importantly, are responsive to the needs of our end users.They're...

  • Back End Engineer

    3 weeks ago


    San Francisco, California, United States Bento for Business Full time

    At Bento, we believe there's a better way for businesses to manage and control employee expenses. This is a rare opportunity to get in on the ground floor of the next big fintech startup that's backed by top tier VCs and founders with a proven track record of success.If you are a talented engineer who thrives in a fast-growing environment, and if you are...


  • San Francisco, California, United States Imbue Full time

    Summary: Much of the work done at Imbue is effectively pure software engineering. Their perspective is that even machine learning research ends up being about 90% software engineering, so even without any prior machine learning knowledge, there is plenty to contribute as a normal software engineer. Even most of Imbue's machine learning research tends towards...

  • Software Engineer

    5 days ago


    San Francisco, California, United States PIQC Institute of Quality Full time

    Company DescriptionSmartRecruiters is a values-driven, global-minded, and well-funded tech employer on a mission to connect people to jobs at scale. As a global leader in enterprise recruitment software, SmartRecruiters offers a cloud-based global Hiring Success platform that allows teams to attract, select, and hire the best talent. 4,000 companies...


  • San Francisco, California, United States Methid Full time

    AI-powered biology automation.Methid is searching for scientists and engineers to be part of the founding team and work on automating life-science methodology.Methid is changing the way biology and medical research are done, right at the edge of bits and atoms. Life-science research and diagnostics are more tedious, laborious, and error-prone than they need...


  • San Francisco, California, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...


  • San Francisco, California, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...