Automate with Python – Full Course for Beginners

Introduction

This tutorial will guide you through automating tasks using Python, drawing from the comprehensive course by Frank Andrade. You'll learn how to automate various repetitive tasks, such as extracting data from websites, creating Excel reports, and sending messages. By the end, you will be equipped with practical skills and knowledge to streamline your workflow.

Step 1: Extract Tables from Websites

Extracting Data

• Use Python libraries like requests and BeautifulSoup for web scraping.
• Example code to extract a table:

  import requests
  from bs4 import BeautifulSoup
  
  url = 'https://example.com'
  response = requests.get(url)
  soup = BeautifulSoup(response.content, 'html.parser')
  
  table = soup.find('table')
  rows = table.find_all('tr')
  
  for row in rows:
      columns = row.find_all('td')
      data = [column.text for column in columns]
      print(data)
  

Extracting CSV Files

• Use pandas to read tables and save them as CSV:

  import pandas as pd
  
  df = pd.read_html(url)
  df[0].to_csv('output.csv', index=False)
  

Extracting Tables from PDFs

• Use tabula-py to extract tables from PDF files:

  from tabula import read_pdf
  
  df = read_pdf('file.pdf', pages='all')
  df.to_csv('output.csv', index=False)
  

Step 2: Web Automation and Web Scraping

Understanding HTML Basics

• Learn about HTML tags and tree structure to navigate web pages effectively.
• Familiarize yourself with XPath for locating elements.

Using XPath

• Practice XPath syntax, functions, and operators to extract data from HTML:

  //tag[@attribute='value']
  

Testing XPath

• Use browser developer tools to test your XPath queries.

Step 3: Automate the News

Installing Necessary Tools

• Install Selenium and ChromeDriver for web automation:

  pip install selenium
  

Creating the Driver

• Create a driver instance to control the browser:

  from selenium import webdriver
  
  driver = webdriver.Chrome(executable_path='path/to/chromedriver')
  driver.get('https://newswebsite.com')
  

Finding Elements and Exporting Data

• Use find_element_by_xpath to locate news elements:

  headlines = driver.find_elements_by_xpath('//h2')
  for headline in headlines:
      print(headline.text)
  

• Export the collected data to a CSV file:

  import csv
  
  with open('news.csv', 'w', newline='') as file:
      writer = csv.writer(file)
      writer.writerows(headline_list)
  

Running Scripts Automatically

• Convert your Python script to an executable:

  pyinstaller --onefile script.py
  

• Schedule the script using crontab on macOS:

  crontab -e
  # Add a line like this to run every day at 8 am
  0 8 * * * /path/to/your/script
  

Step 4: Automate Excel Reports

Creating Pivot Tables

• Use pandas to create pivot tables:

  pivot_table = df.pivot_table(values='value', index='index', columns='column')
  

Adding Charts and Formatting

• Use openpyxl to add bar charts and format cells within Excel:

  from openpyxl import Workbook
  from openpyxl.chart import BarChart, Reference
  
  wb = Workbook()
  ws = wb.active
  
  # Add data and create a chart
  chart = BarChart()
  data = Reference(ws, min_col=1, min_row=1, max_col=3, max_row=10)
  chart.add_data(data)
  ws.add_chart(chart, "E5")
  

Generating Reports with One Click

• Combine your scripts into a single executable for easy report generation.

Step 5: Automate WhatsApp

• Use the pywhatkit library to send messages via WhatsApp:

  import pywhatkit as kit
  
  kit.sendwhatmsg('+1234567890', 'Hello!', 15, 0)  # Sends message at 3:00 PM
  

Conclusion

By following these steps, you can automate various tasks using Python, from web scraping to generating Excel reports and messaging. Explore additional libraries and tools mentioned to further enhance your automation skills. Consider diving deeper into the provided resources for more advanced projects and techniques. Happy automating!