Complete Python Roadmap (Beginner to Advanced)

Stage 1: Python Basics

1. Print Function

The print() function displays output on the screen.

print("Hello, World!")

Output:

Hello, World!

Examples:

print(10)
print(10 + 20)
print("Python")

2. Input Function

The input() function takes input from the user.

name = input("Enter your name: ")
print("Hello", name)

Example:

Enter your name: John
Hello John

3. Variables

Variables store data.

name = "Alice"
age = 25
height = 5.8

print(name)
print(age)
print(height)

Rules:

Start with a letter or _
Cannot start with a number
Case-sensitive

Correct:

age = 20
_age = 30
userName = "John"

Wrong:

2age = 20

4. Data Types

name = "Python"      # String
age = 20             # Integer
price = 99.99        # Float
is_student = True    # Boolean

Check type:

print(type(name))
print(type(age))

5. Comments

Single-line comment

# This is a comment

Multi-line

"""
This is
a multi-line
comment
"""

Stage 2: Operators

Arithmetic Operators

a = 10
b = 3

print(a + b)
print(a - b)
print(a * b)
print(a / b)
print(a // b)
print(a % b)
print(a ** b)

Comparison Operators

print(5 == 5)
print(5 != 4)
print(5 > 2)
print(5 < 7)

Logical Operators

print(True and False)
print(True or False)
print(not True)

Assignment Operators

x = 5

x += 3
x -= 2
x *= 4
x /= 2

Stage 3: Type Conversion

age = "20"

age = int(age)

print(age + 5)

Other conversions:

float()
str()
bool()
list()
tuple()
set()

Stage 4: Strings

name = "Python"

print(name[0])
print(name[-1])
print(len(name))

Common functions:

name.upper()

name.lower()

name.title()

name.capitalize()

name.replace("P", "J")

name.find("t")

name.count("o")

name.strip()

name.split()

",".join(["A","B","C"])

String Formatting

name = "John"
age = 25

print(f"My name is {name} and I am {age}")

Stage 5: Conditional Statements

if

age = 18

if age >= 18:
    print("Adult")

if else

age = 16

if age >= 18:
    print("Adult")
else:
    print("Minor")

if elif else

marks = 85

if marks >= 90:
    print("A")

elif marks >= 80:
    print("B")

elif marks >= 70:
    print("C")

else:
    print("Fail")

Stage 6: Loops

for Loop

for i in range(5):
    print(i)

Output

while Loop

count = 1

while count <= 5:
    print(count)
    count += 1

break

for i in range(10):

    if i == 5:
        break

    print(i)

continue

for i in range(5):

    if i == 2:
        continue

    print(i)

Stage 7: Functions (Very Important)

A function is a reusable block of code.

Simple Function

def greet():
    print("Hello")

greet()

Function with Parameter

def greet(name):
    print("Hello", name)

greet("Alice")

Function with Return

def add(a, b):
    return a + b

result = add(5, 3)

print(result)

Default Parameter

def greet(name="Guest"):
    print("Hello", name)

greet()
greet("John")

Keyword Arguments

def student(name, age):
    print(name, age)

student(age=20, name="John")

Variable-Length Arguments

def add_all(*numbers):
    total = 0

    for num in numbers:
        total += num

    return total

print(add_all(1,2,3,4))

Keyword Variable Arguments

def details(**info):
    print(info)

details(name="John", age=25)

Lambda Function

square = lambda x: x*x

print(square(5))

Recursive Function

def factorial(n):

    if n == 1:
        return 1

    return n * factorial(n-1)

print(factorial(5))

Stage 8: Lists

numbers = [10,20,30]

numbers.append(40)

numbers.insert(1,15)

numbers.remove(20)

numbers.pop()

numbers.sort()

numbers.reverse()

print(numbers)

Stage 9: Tuples

colors = ("Red","Blue","Green")

print(colors[0])

Stage 10: Sets

numbers = {1,2,3,4}

numbers.add(5)

numbers.remove(2)

print(numbers)

Stage 11: Dictionaries

student = {
    "name":"John",
    "age":20,
    "city":"NY"
}

print(student["name"])

student["age"] = 21

print(student.keys())
print(student.values())

Stage 12: File Handling

file = open("test.txt","w")

file.write("Hello")

file.close()

Reading:

file = open("test.txt","r")

print(file.read())

file.close()

A better approach is using with:

with open("test.txt", "r") as file:
    print(file.read())

Stage 13: Exception Handling

try:
    x = 10/0

except ZeroDivisionError:
    print("Cannot divide by zero")

finally:
    print("Finished")

Stage 14: Object-Oriented Programming (OOP)

class Student:

    def __init__(self, name):
        self.name = name

    def display(self):
        print(self.name)

s = Student("John")

s.display()

Topics to learn next:

Classes
Objects
Constructors (__init__)
Inheritance
Polymorphism
Encapsulation
Abstraction

Stage 15: Modules

import math

print(math.sqrt(25))

Stage 16: Popular Built-in Functions

Function	Use
`print()`	Display output
`input()`	Read user input
`len()`	Length of an object
`type()`	Data type
`range()`	Generate numbers
`int()`	Convert to integer
`float()`	Convert to float
`str()`	Convert to string
`list()`	Create a list
`tuple()`	Create a tuple
`set()`	Create a set
`dict()`	Create a dictionary
`sum()`	Sum numbers
`max()`	Largest value
`min()`	Smallest value
`sorted()`	Return a sorted list
`abs()`	Absolute value
`round()`	Round a number
`enumerate()`	Index + value in loops
`zip()`	Combine iterables
`map()`	Apply a function to items
`filter()`	Filter items by a condition
`any()`	`True` if any element is true
`all()`	`True` if all elements are true

Stage 17: Advanced Python

List comprehensions
Dictionary comprehensions
Decorators
Generators (yield)
Iterators
Closures
Regular expressions (re)
collections module
itertools
functools
Multithreading
Multiprocessing
Async programming (async/await)
Virtual environments
Package management (pip)
Testing with unittest and pytest
Working with APIs (requests)
Database access (SQLite, PostgreSQL)
NumPy, Pandas, Matplotlib
Flask or Django for web development

Week 1: Basics, variables, data types, operators, input/output
Week 2: Strings, conditionals, loops, and functions
Week 3: Lists, tuples, sets, dictionaries, file handling
Week 4: OOP, modules, exceptions, and small projects
Week 5+: Advanced Python and specialization (web, data science, automation, AI, etc.)

Python `if` Statement

An if statement is used to make decisions in a program.

Syntax:

if condition:
    # code to execute if condition is True

If the condition is True, the code inside the if block runs. If it is False, Python skips it.

Example 1: Simple `if`

age = 20

if age >= 18:
    print("You are an adult.")

Output:

You are an adult.

Explanation:

age is 20.
20 >= 18 is True.
So Python prints the message.

Example 2: Condition is False

age = 15

if age >= 18:
    print("You are an adult.")

print("Program finished.")

Output:

Program finished.

Since 15 >= 18 is False, the print() inside the if block is skipped.

Comparison Operators Used in `if`

Operator	Meaning	Example
`==`	Equal to	`x == 10`
`!=`	Not equal to	`x != 10`
`>`	Greater than	`x > 5`
`<`	Less than	`x < 5`
`>=`	Greater than or equal to	`x >= 18`
`<=`	Less than or equal to	`x <= 100`

Example 3: Using `==`

password = "python123"

if password == "python123":
    print("Access granted")

Output:

Access granted

Example 4: Using `!=`

number = 5

if number != 10:
    print("Number is not 10")

Output:

Number is not 10

Using User Input

age = int(input("Enter your age: "))

if age >= 18:
    print("You can vote.")

Example Input:

Enter your age: 21

Output:

You can vote.

Using Strings

name = input("Enter your name: ")

if name == "Alice":
    print("Welcome Alice!")

Multiple Statements Inside `if`

marks = 95

if marks >= 90:
    print("Excellent!")
    print("Grade: A")
    print("Keep it up!")

All three print() statements execute because the condition is True.

Indentation in Python

Python uses indentation (spaces) to define blocks of code.

✅ Correct:

age = 20

if age >= 18:
    print("Adult")

❌ Incorrect:

age = 20

if age >= 18:
print("Adult")

This causes an IndentationError.

Nested `if`

An if statement can be placed inside another if.

age = 20
has_id = True

if age >= 18:
    if has_id:
        print("Entry allowed")

Output:

Entry allowed

Practice Programs

Program 1: Check Positive Number

number = int(input("Enter a number: "))

if number > 0:
    print("Positive number")

Program 2: Check Even Number

number = int(input("Enter a number: "))

if number % 2 == 0:
    print("Even number")

Program 3: Check Password

password = input("Enter password: ")

if password == "admin123":
    print("Login successful")

Program 4: Check Passing Marks

marks = int(input("Enter marks: "))

if marks >= 40:
    print("Pass")

Key Points

if is used to make decisions.
The condition must evaluate to True or False.
Indentation is required.
You can use comparison operators (==, !=, >, <, >=, <=).
You can use variables, user input, and expressions in conditions.

Exercises

Try writing these programs yourself:

Check if a number is greater than 100.
Check if a person’s age is at least 21.
Check if a character is 'A'.
Check if a number is divisible by 5.
Check if a student’s marks are 90 or above.

Python `if...else` Statement

The if...else statement is used when you want your program to perform one action if a condition is True and another action if it is False.

Syntax

if condition:
    # Executes if condition is True
else:
    # Executes if condition is False

Flow Diagram

        Condition
            │
      ┌─────┴─────┐
      │           │
    True       False
      │           │
   if block   else block

Example 1: Check Age

age = 20

if age >= 18:
    print("You are eligible to vote.")
else:
    print("You are not eligible to vote.")

Output

You are eligible to vote.

Explanation

age = 20
20 >= 18 is True
Python executes the if block.
The else block is skipped.

Example 2: Condition is False

age = 15

if age >= 18:
    print("Adult")
else:
    print("Minor")

Output

Minor

Example 3: Even or Odd

number = 9

if number % 2 == 0:
    print("Even")
else:
    print("Odd")

Output

Odd

Explanation

9 % 2 equals 1, so the condition is False.

Example 4: Positive or Negative

number = -5

if number >= 0:
    print("Positive")
else:
    print("Negative")

Example 5: Password Check

password = input("Enter password: ")

if password == "python123":
    print("Access Granted")
else:
    print("Wrong Password")

Example

Enter password: hello
Wrong Password

Example 6: Largest of Two Numbers

a = 30
b = 25

if a > b:
    print("a is larger")
else:
    print("b is larger")

Example 7: Check Divisibility

number = int(input("Enter a number: "))

if number % 5 == 0:
    print("Divisible by 5")
else:
    print("Not divisible by 5")

Example 8: Boolean Values

logged_in = True

if logged_in:
    print("Welcome!")
else:
    print("Please log in.")

Since logged_in is True, Python executes the if block.

Example 9: Compare Strings

color = input("Enter a color: ")

if color == "red":
    print("Stop")
else:
    print("Go")

Nested `if...else`

age = 20
has_id = True

if age >= 18:
    if has_id:
        print("Entry Allowed")
    else:
        print("ID Required")
else:
    print("Under Age")

Common Mistake

❌ Incorrect

age = 18

if age >= 18:
print("Adult")
else:
print("Minor")

This gives an IndentationError.

✅ Correct

age = 18

if age >= 18:
    print("Adult")
else:
    print("Minor")

Practice Programs

Program 1: Pass or Fail

marks = int(input("Enter marks: "))

if marks >= 40:
    print("Pass")
else:
    print("Fail")

Program 2: Check Driving Eligibility

age = int(input("Enter age: "))

if age >= 18:
    print("Eligible for driving license")
else:
    print("Not eligible")

Program 3: Login System

username = input("Username: ")
password = input("Password: ")

if username == "admin" and password == "1234":
    print("Login Successful")
else:
    print("Login Failed")

Program 4: Largest Number

a = int(input("Enter first number: "))
b = int(input("Enter second number: "))

if a > b:
    print(a, "is larger")
else:
    print(b, "is larger")

Program 5: Positive or Negative

number = int(input("Enter a number: "))

if number >= 0:
    print("Positive")
else:
    print("Negative")

Summary

if checks a condition.
else runs only when the if condition is False.
Exactly one block (if or else) executes.
Proper indentation is required.

Exercises

Try solving these on your own:

Check whether a number is positive or negative.
Check whether a person can vote (age ≥ 18).
Check if a number is divisible by 10.
Compare two numbers and print the larger one.
Create a simple username and password checker.

Excellent! Now let’s learn one of the most important concepts in Python.

Chapter 6: `for` Loop

What is a Loop?

A loop is used to execute the same block of code multiple times.

Instead of writing:

print("Hello")
print("Hello")
print("Hello")
print("Hello")
print("Hello")

You can write:

for i in range(5):
    print("Hello")

Output

Hello
Hello
Hello
Hello
Hello

What is a `for` Loop?

A for loop is used to iterate over a sequence such as:

Numbers
Strings
Lists
Tuples
Dictionaries
Sets

Syntax

for variable in sequence:
    # code

Example:

for i in range(5):
    print(i)

Understanding `range()`

range() generates a sequence of numbers.

1. `range(stop)`

for i in range(5):
    print(i)

Output

Notice that 5 is not included.

2. `range(start, stop)`

for i in range(2, 8):
    print(i)

Output

3. `range(start, stop, step)`

for i in range(1, 11, 2):
    print(i)

Output

Here:

Start = 1
Stop = 11 (not included)
Step = 2

Printing Numbers

for i in range(1, 11):
    print(i)

Output

Print Squares

for i in range(1, 6):
    print(i ** 2)

Output

Print Cubes

for i in range(1, 6):
    print(i ** 3)

Output

Loop Through a String

word = "Python"

for letter in word:
    print(letter)

Output

P
y
t
h
o
n

Each iteration gives one character.

Loop Through a List

fruits = ["Apple", "Banana", "Orange"]

for fruit in fruits:
    print(fruit)

Output

Apple
Banana
Orange

Loop Through a Tuple

numbers = (10, 20, 30)

for n in numbers:
    print(n)

Loop Through a Set

colors = {"Red", "Blue", "Green"}

for color in colors:
    print(color)

Note: Sets are unordered, so the output order may vary.

Loop Through a Dictionary

student = {
    "name": "John",
    "age": 20,
    "city": "New York"
}

for key in student:
    print(key)

Output

name
age
city

To print both keys and values:

for key, value in student.items():
    print(key, ":", value)

Output

name : John
age : 20
city : New York

Multiplication Table

number = 5

for i in range(1, 11):
    print(number, "x", i, "=", number * i)

Output

5 x 1 = 5
5 x 2 = 10
...
5 x 10 = 50

Sum of Numbers

total = 0

for i in range(1, 6):
    total += i

print(total)

Output

Explanation:

0 + 1 = 1
1 + 2 = 3
3 + 3 = 6
6 + 4 = 10
10 + 5 = 15

Find Even Numbers

for i in range(1, 11):
    if i % 2 == 0:
        print(i)

Output

Find Odd Numbers

for i in range(1, 11):
    if i % 2 != 0:
        print(i)

Output

Reverse Counting

for i in range(10, 0, -1):
    print(i)

Output

Nested `for` Loop

A loop inside another loop is called a nested loop.

for i in range(3):
    for j in range(2):
        print(i, j)

Output

Pattern Printing

for i in range(5):
    print("*" * (i + 1))

Output

*
**
***
****
*****

Using `enumerate()`

enumerate() returns both the index and the value.

fruits = ["Apple", "Banana", "Orange"]

for index, fruit in enumerate(fruits):
    print(index, fruit)

Output

0 Apple
1 Banana
2 Orange

Using `zip()`

zip() combines multiple sequences.

names = ["Alice", "Bob", "Charlie"]
scores = [90, 85, 95]

for name, score in zip(names, scores):
    print(name, score)

Output

Alice 90
Bob 85
Charlie 95

Common Mistakes

❌ Forgetting the colon:

for i in range(5)
    print(i)

This causes a SyntaxError.

✅ Correct:

for i in range(5):
    print(i)

Practice Programs

Program 1: Print 1–20

for i in range(1, 21):
    print(i)

Program 2: Sum of 1–100

total = 0

for i in range(1, 101):
    total += i

print(total)

Program 3: Print Each Letter

name = input("Enter your name: ")

for ch in name:
    print(ch)

Program 4: Count Down

for i in range(5, 0, -1):
    print(i)

print("Blast Off!")

Summary

for loops repeat code.
range() generates numbers.
You can loop through strings, lists, tuples, sets, and dictionaries.
Nested loops allow more complex repetition.
enumerate() provides indexes.
zip() combines multiple sequences.

Exercises

Print numbers from 1 to 50.
Print only multiples of 3 from 1 to 30.
Print the multiplication table of any number entered by the user.
Count the number of vowels in a word.
Print this pattern:

*
**
***
****
*****

Chapter 7: Python `while` Loop

What is a `while` Loop?

A while loop repeatedly executes a block of code as long as a condition is True.

Unlike a for loop (where we usually know the number of repetitions), a while loop is useful when we don’t know exactly how many times we need to repeat.

Syntax

while condition:
    # code to execute

Example:

id = "1abc23"
count = 1

while count <= 5:
    print(count)
    count += 1

Output:

How `while` Loop Works

Example:

count = 1

while count <= 3:
    print("Hello")
    count = count + 1

Step by step:

count = 1 → print Hello → count becomes 2
count = 2 → print Hello → count becomes 3
count = 3 → print Hello → count becomes 4
count = 4 → condition False → stop

Example 1: Print Numbers 1 to 10

i = 1

while i <= 10:
    print(i)
    i += 1

Output:

Example 2: Countdown

number = 10

while number >= 1:
    print(number)
    number -= 1

Output:

Example 3: Sum of Numbers

total = 0
number = 1

while number <= 5:
    total += number
    number += 1

print(total)

Output:

Example 4: User Input Loop

A while loop can keep asking until the user enters the correct value.

password = ""

while password != "python123":
    password = input("Enter password: ")

print("Login successful")

Example:

Enter password: hello
Enter password: test
Enter password: python123

Login successful

Infinite Loop

An infinite loop never stops because the condition is always True.

Example:

while True:
    print("Hello")

Output:

Hello
Hello
Hello
...

To stop it, use break.

`break` Statement

break immediately exits the loop.

Example:

i = 1

while i <= 10:

    if i == 5:
        break

    print(i)
    i += 1

Output:

When i becomes 5, the loop stops.

Example: Search Number

numbers = [10,20,30,40,50]

i = 0

while i < len(numbers):

    if numbers[i] == 30:
        print("Found")
        break

    i += 1

Output:

Found

`continue` Statement

continue skips the current iteration and moves to the next one.

Example:

i = 0

while i < 5:

    i += 1

    if i == 3:
        continue

    print(i)

Output:

Number 3 was skipped.

`pass` Statement

pass does nothing.

It is used as a placeholder.

Example:

while True:
    pass

Example with condition:

age = 18

if age >= 18:
    pass
else:
    print("Not allowed")

Nested `while` Loop

A while loop inside another while loop.

Example:

i = 1

while i <= 3:

    j = 1

    while j <= 3:
        print(i, j)
        j += 1

    i += 1

Output:

Practical Programs

1. Even Numbers

i = 1

while i <= 20:

    if i % 2 == 0:
        print(i)

    i += 1

2. Odd Numbers

i = 1

while i <= 20:

    if i % 2 != 0:
        print(i)

    i += 1

3. Multiplication Table

num = int(input("Enter number: "))

i = 1

while i <= 10:

    print(num, "x", i, "=", num*i)

    i += 1

4. Count Digits in a Number

number = 12345

count = 0

while number > 0:

    number = number // 10
    count += 1

print(count)

Output:

5. Reverse a Number

number = 1234
reverse = 0

while number > 0:

    digit = number % 10

    reverse = reverse * 10 + digit

    number = number // 10

print(reverse)

Output:

6. Guessing Game

secret = 7

guess = 0

while guess != secret:

    guess = int(input("Guess number: "))

    if guess < secret:
        print("Too low")

    elif guess > secret:
        print("Too high")

    else:
        print("Correct!")

Difference Between `for` and `while`

`for` loop	`while` loop
Used when number of repetitions is known	Used when repetitions are unknown
Works with sequences	Works with conditions
Uses `range()` often	Uses a condition
Easier for counting	Better for user input loops

Example:

for loop

for i in range(5):
    print(i)

while loop

i = 0

while i < 5:
    print(i)
    i += 1

Both give:

Practice Exercises

Print numbers from 100 to 1 using while.
Create a program that keeps asking for a number until the user enters 0.
Find the factorial of a number using while.
Create a simple ATM menu using while.
Create a program to check if a number is a palindrome.

Great! Now we start one of the most important Python topics.

Chapter 9: Python Lists

What is a List?

A list is a collection used to store multiple values in a single variable.

Lists can store:

Numbers
Strings
Mixed data types
Other lists

Lists are:

Ordered (items have positions)
Changeable (you can modify them)
Allow duplicates

Creating a List

Example 1: List of Numbers

numbers = [10, 20, 30, 40]

print(numbers)

Output:

[10, 20, 30, 40]

Example 2: List of Strings

fruits = ["Apple", "Banana", "Orange"]

print(fruits)

Output:

['Apple', 'Banana', 'Orange']

Example 3: Mixed Data Types

data = ["John", 25, 5.8, True]

print(data)

Output:

['John', 25, 5.8, True]

Accessing List Items

Each item has an index number.

Example:

fruits = ["Apple", "Banana", "Orange"]

print(fruits[0])
print(fruits[1])
print(fruits[2])

Output:

Apple
Banana
Orange

Remember:

Index:
0 → First item
1 → Second item
2 → Third item

Negative Indexing

Negative indexes count from the end.

fruits = ["Apple", "Banana", "Orange"]

print(fruits[-1])
print(fruits[-2])
print(fruits[-3])

Output:

Orange
Banana
Apple

Changing List Items

Lists are mutable (changeable).

Example:

fruits = ["Apple", "Banana", "Orange"]

fruits[1] = "Mango"

print(fruits)

Output:

['Apple', 'Mango', 'Orange']

Adding Items to a List

1. `append()`

Adds an item at the end.

fruits = ["Apple", "Banana"]

fruits.append("Orange")

print(fruits)

Output:

['Apple', 'Banana', 'Orange']

2. `insert()`

Adds an item at a specific position.

Syntax:

list.insert(index, value)

Example:

fruits = ["Apple", "Banana"]

fruits.insert(1, "Mango")

print(fruits)

Output:

['Apple', 'Mango', 'Banana']

3. `extend()`

Adds multiple items.

a = [1,2,3]

b = [4,5,6]

a.extend(b)

print(a)

Output:

[1,2,3,4,5,6]

Removing Items from List

1. `remove()`

Removes a specific value.

fruits = ["Apple","Banana","Orange"]

fruits.remove("Banana")

print(fruits)

Output:

['Apple', 'Orange']

2. `pop()`

Removes item by index.

fruits = ["Apple","Banana","Orange"]

fruits.pop(1)

print(fruits)

Output:

['Apple','Orange']

Without index:

fruits.pop()

Removes the last item.

3. `del`

Deletes an item.

numbers = [10,20,30]

del numbers[1]

print(numbers)

Output:

[10,30]

4. `clear()`

Removes everything.

numbers = [1,2,3]

numbers.clear()

print(numbers)

Output:

[]

List Length

Use len():

fruits = ["Apple","Banana","Orange"]

print(len(fruits))

Output:

Checking Items

Use in.

fruits = ["Apple","Banana","Orange"]

if "Apple" in fruits:
    print("Found")

Output:

Found

Loop Through a List

Using for loop

fruits = ["Apple","Banana","Orange"]

for fruit in fruits:
    print(fruit)

Output:

Apple
Banana
Orange

Using while loop

fruits = ["Apple","Banana","Orange"]

i = 0

while i < len(fruits):

    print(fruits[i])

    i += 1

List Slicing

Slicing gets a part of a list.

Syntax:

list[start:end]

Example:

numbers = [10,20,30,40,50]

print(numbers[1:4])

Output:

[20,30,40]

More Slicing Examples

numbers = [1,2,3,4,5]

print(numbers[:3])

Output:

[1,2,3]

print(numbers[2:])

Output:

[3,4,5]

Reverse list:

numbers = [1,2,3,4,5]

print(numbers[::-1])

Output:

[5,4,3,2,1]

List Sorting

`sort()`

numbers = [5,2,8,1]

numbers.sort()

print(numbers)

Output:

[1,2,5,8]

Descending order:

numbers.sort(reverse=True)

print(numbers)

Output:

[8,5,2,1]

Reverse List

numbers = [1,2,3,4]

numbers.reverse()

print(numbers)

Output:

[4,3,2,1]

Copying Lists

Wrong way:

a = [1,2,3]

b = a

Both point to the same list.

Correct:

a = [1,2,3]

b = a.copy()

print(b)

Nested Lists

A list inside another list.

students = [
    ["John",20],
    ["Alice",22],
    ["Bob",21]
]

print(students[0])

Output:

['John',20]

Access inner item:

print(students[0][0])

Output:

John

Useful List Functions

Function	Use
`append()`	Add item at end
`insert()`	Add item at position
`extend()`	Add multiple items
`remove()`	Remove value
`pop()`	Remove by index
`clear()`	Empty list
`sort()`	Sort list
`reverse()`	Reverse list
`copy()`	Copy list
`len()`	Count items
`max()`	Largest value
`min()`	Smallest value
`sum()`	Total values

Practical Programs

1. Find Largest Number

numbers = [10,50,20,90,30]

print(max(numbers))

Output:

2. Sum of List

numbers = [10,20,30]

total = sum(numbers)

print(total)

Output:

3. Count Even Numbers

numbers = [1,2,3,4,5,6]

count = 0

for n in numbers:

    if n % 2 == 0:
        count += 1

print(count)

Output:

Practice Exercises

Create a list of 10 numbers and print all items.
Find the largest and smallest number in a list.
Remove duplicate items from a list.
Reverse a list without using reverse().
Create a shopping cart program using a list.

Chapter 10: Advanced List Methods & List Comprehension

Lists are very powerful in Python. In this chapter, we learn advanced ways to create, modify, and process lists.

1. List Methods (Detailed)

`append()`

Adds one item at the end of a list.

numbers = [1, 2, 3]

numbers.append(4)

print(numbers)

Output:

[1, 2, 3, 4]

`insert()`

Adds an item at a specific position.

names = ["John", "Bob"]

names.insert(1, "Alice")

print(names)

Output:

['John', 'Alice', 'Bob']

`extend()`

Adds multiple items from another list.

a = [1, 2, 3]

b = [4, 5, 6]

a.extend(b)

print(a)

Output:

[1, 2, 3, 4, 5, 6]

`remove()`

Removes the first matching value.

numbers = [10,20,30,20]

numbers.remove(20)

print(numbers)

Output:

[10,30,20]

`pop()`

Removes and returns an item.

numbers = [10,20,30]

x = numbers.pop()

print(x)
print(numbers)

Output:

30
[10,20]

`index()`

Finds the position of an item.

fruits = ["Apple","Banana","Orange"]

print(fruits.index("Banana"))

Output:

`count()`

Counts how many times an item appears.

numbers = [1,2,2,3,2]

print(numbers.count(2))

Output:

`sort()`

Sorts the list.

Ascending:

numbers = [5,2,8,1]

numbers.sort()

print(numbers)

Output:

[1,2,5,8]

Descending:

numbers.sort(reverse=True)

Output:

[8,5,2,1]

`reverse()`

Reverses the list.

numbers = [1,2,3,4]

numbers.reverse()

print(numbers)

Output:

[4,3,2,1]

2. List Copying

Normal Copy Problem

a = [1,2,3]

b = a

b.append(4)

print(a)

Output:

[1,2,3,4]

Why?

Because b and a point to the same list.

Correct Copy

Using copy():

a = [1,2,3]

b = a.copy()

b.append(4)

print(a)
print(b)

Output:

[1,2,3]
[1,2,3,4]

3. List Comprehension

What is List Comprehension?

List comprehension is a shorter way to create lists.

Normal way:

numbers = []

for i in range(1,6):
    numbers.append(i)

print(numbers)

Output:

[1,2,3,4,5]

Using list comprehension:

numbers = [i for i in range(1,6)]

print(numbers)

Output:

[1,2,3,4,5]

List Comprehension Syntax

new_list = [expression for item in iterable]

Example:

squares = [x*x for x in range(1,6)]

print(squares)

Output:

[1,4,9,16,25]

4. List Comprehension with Condition

Syntax:

new_list = [expression for item in iterable if condition]

Example: Even Numbers

Normal:

even = []

for i in range(1,11):

    if i % 2 == 0:
        even.append(i)

print(even)

Using comprehension:

even = [i for i in range(1,11) if i % 2 == 0]

print(even)

Output:

[2,4,6,8,10]

5. Convert Strings

names = ["john","alice","bob"]

upper_names = [name.upper() for name in names]

print(upper_names)

Output:

['JOHN','ALICE','BOB']

6. Filter Numbers

numbers = [10,15,20,25,30]

result = [x for x in numbers if x > 20]

print(result)

Output:

[25,30]

7. If Else in List Comprehension

Syntax:

[value_if_true if condition else value_if_false for item in list]

Example:

numbers = [1,2,3,4,5]

result = [
    "Even" if x%2==0 else "Odd"
    for x in numbers
]

print(result)

Output:

['Odd','Even','Odd','Even','Odd']

8. Nested List Comprehension

Example:

Create multiplication table:

table = [
    [i*j for j in range(1,6)]
    for i in range(1,6)
]

print(table)

Output:

[
[1,2,3,4,5],
[2,4,6,8,10],
[3,6,9,12,15]
]

9. Flatten a Nested List

Example:

matrix = [
    [1,2,3],
    [4,5,6],
    [7,8,9]
]

flat = [
    num
    for row in matrix
    for num in row
]

print(flat)

Output:

[1,2,3,4,5,6,7,8,9]

10. Useful List Functions

`len()`

numbers = [1,2,3]

print(len(numbers))

Output:

`max()`

numbers = [10,50,20]

print(max(numbers))

Output:

`min()`

print(min(numbers))

Output:

`sum()`

print(sum(numbers))

Output:

Practical Programs

1. Remove Duplicates

numbers = [1,2,2,3,4,4,5]

unique = []

for n in numbers:

    if n not in unique:
        unique.append(n)

print(unique)

Output:

[1,2,3,4,5]

2. Find Common Elements

a = [1,2,3,4]

b = [3,4,5,6]

common = [x for x in a if x in b]

print(common)

Output:

[3,4]

3. Word Length

words = ["Python","Java","C"]

lengths = [len(word) for word in words]

print(lengths)

Output:

[6,4,1]

Practice Exercises

Create a list of squares from 1 to 20.
Extract only vowels from a word.
Convert all names in a list to uppercase.
Remove all negative numbers from a list.
Flatten this list:

[[1,2],[3,4],[5,6]]

Create a list of numbers divisible by 5 from 1 to 100.

Chapter 11: Python Tuples

What is a Tuple?

A tuple is a collection used to store multiple values in a single variable.

A tuple is similar to a list, but the main difference is:

List → Can be changed (mutable)
Tuple → Cannot be changed (immutable)

Creating a Tuple

Example 1: Tuple of Numbers

numbers = (10, 20, 30, 40)

print(numbers)

Output:

(10, 20, 30, 40)

Example 2: Tuple of Strings

fruits = ("Apple", "Banana", "Orange")

print(fruits)

Output:

('Apple', 'Banana', 'Orange')

Example 3: Mixed Data Types

data = ("John", 25, 5.8, True)

print(data)

Output:

('John', 25, 5.8, True)

Creating a Tuple Without Parentheses

Parentheses are optional.

colors = "Red", "Blue", "Green"

print(colors)

Output:

('Red', 'Blue', 'Green')

Single Item Tuple

A tuple with one item needs a comma.

Correct:

number = (10,)

print(type(number))

Output:

<class 'tuple'>

Wrong:

number = (10)

print(type(number))

Output:

<class 'int'>

Accessing Tuple Items

Tuples use indexes like lists.

fruits = ("Apple", "Banana", "Orange")

print(fruits[0])
print(fruits[1])
print(fruits[2])

Output:

Apple
Banana
Orange

Negative Indexing

fruits = ("Apple", "Banana", "Orange")

print(fruits[-1])

Output:

Orange

Tuple Slicing

Syntax:

tuple[start:end]

Example:

numbers = (10,20,30,40,50)

print(numbers[1:4])

Output:

(20,30,40)

Tuples are Immutable

You cannot change tuple values.

Example:

numbers = (10,20,30)

numbers[1] = 50

Output:

TypeError

Because tuples cannot be modified.

Converting Tuple to List

If you need to modify a tuple, convert it to a list.

numbers = (10,20,30)

list_numbers = list(numbers)

list_numbers[1] = 50

numbers = tuple(list_numbers)

print(numbers)

Output:

(10,50,30)

Tuple Methods

Tuples have only two built-in methods because they cannot be changed.

1. `count()`

Counts how many times a value appears.

numbers = (1,2,2,3,2)

print(numbers.count(2))

Output:

2. `index()`

Returns the position of the first occurrence.

numbers = (10,20,30,40)

print(numbers.index(30))

Output:

Loop Through a Tuple

Using for loop

fruits = ("Apple","Banana","Orange")

for fruit in fruits:
    print(fruit)

Output:

Apple
Banana
Orange

Using while loop

numbers = (10,20,30)

i = 0

while i < len(numbers):

    print(numbers[i])

    i += 1

Tuple Packing

Putting multiple values into a tuple.

student = "John", 20, "Python"

print(student)

Output:

('John',20,'Python')

Tuple Unpacking

Taking values out of a tuple.

student = ("John",20,"Python")

name, age, course = student

print(name)
print(age)
print(course)

Output:

John
20
Python

Swapping Variables Using Tuple

Without temporary variable:

a = 10
b = 20

a, b = b, a

print(a)
print(b)

Output:

20
10

Nested Tuples

A tuple can contain other tuples.

students = (
    ("John",20),
    ("Alice",22),
    ("Bob",21)
)

print(students[0])

Output:

('John',20)

Access inner value:

print(students[0][0])

Output:

John

Tuple vs List

Feature	List	Tuple
Syntax	`[ ]`	`( )`
Changeable	Yes	No
Speed	Slower	Faster
Methods	Many	Few
Memory	More	Less
Use	Data that changes	Fixed data

When Should You Use Tuples?

Use tuples when:

Data should not change
You want faster performance
You want to protect data from accidental modification

Examples:

coordinates = (10.5, 20.5)

rgb_color = (255, 0, 0)

days = ("Monday","Tuesday","Wednesday")

Useful Tuple Functions

`len()`

numbers = (1,2,3,4)

print(len(numbers))

Output:

`max()`

numbers = (10,50,20)

print(max(numbers))

Output:

`min()`

print(min(numbers))

Output:

`sum()`

numbers = (10,20,30)

print(sum(numbers))

Output:

Practical Programs

1. Store Student Information

student = ("John", 21, "Python")

name, age, course = student

print("Name:", name)
print("Age:", age)
print("Course:", course)

2. Find Maximum Number

numbers = (5,10,15,20)

print(max(numbers))

3. Count Occurrences

numbers = (1,2,2,3,2,4)

print(numbers.count(2))

Output:

Practice Exercises

Create a tuple of 5 countries and print each country.
Find the length of a tuple.
Count how many times a number appears in a tuple.
Convert a tuple into a list and modify it.
Store student records using nested tuples.

Chapter 12: Python Sets

What is a Set?

A set is a collection used to store multiple values.

Important properties of sets:

✅ Unordered (items have no fixed position)
✅ Mutable (can be changed)
✅ Does not allow duplicate values
✅ Items must be unique

Example:

numbers = {1, 2, 3, 4}

print(numbers)

Output:

{1, 2, 3, 4}

Creating a Set

Example 1: Numbers Set

numbers = {10, 20, 30, 40}

print(numbers)

Example 2: String Set

colors = {"Red", "Blue", "Green"}

print(colors)

Output order may change because sets are unordered.

Example:

{'Green', 'Red', 'Blue'}

Duplicate Values in Set

Sets automatically remove duplicates.

numbers = {1,2,2,3,3,4}

print(numbers)

Output:

{1,2,3,4}

Creating an Empty Set

Important:

This is NOT a set:

empty = {}

print(type(empty))

Output:

<class 'dict'>

Correct way:

empty = set()

print(type(empty))

Output:

<class 'set'>

Accessing Set Items

Sets do not have indexes.

This will not work:

colors = {"Red","Blue","Green"}

print(colors[0])

Output:

TypeError

Loop Through a Set

Use a for loop:

colors = {"Red","Blue","Green"}

for color in colors:
    print(color)

Output:

Red
Blue
Green

(The order can vary.)

Adding Items to a Set

1. `add()`

Adds one item.

colors = {"Red","Blue"}

colors.add("Green")

print(colors)

Output:

{'Red','Blue','Green'}

2. `update()`

Adds multiple items.

numbers = {1,2,3}

numbers.update([4,5,6])

print(numbers)

Output:

{1,2,3,4,5,6}

Removing Items from a Set

1. `remove()`

Removes an item.

colors = {"Red","Blue","Green"}

colors.remove("Blue")

print(colors)

If item does not exist:

colors.remove("Yellow")

It gives:

KeyError

2. `discard()`

Removes an item safely.

colors = {"Red","Blue"}

colors.discard("Yellow")

print(colors)

No error occurs.

3. `pop()`

Removes a random item.

numbers = {10,20,30}

numbers.pop()

print(numbers)

4. `clear()`

Removes all items.

numbers = {1,2,3}

numbers.clear()

print(numbers)

Output:

set()

Set Operations

Sets are mainly useful because of mathematical operations.

1. Union (`|`)

Combines two sets.

Example:

a = {1,2,3}

b = {3,4,5}

print(a | b)

Output:

{1,2,3,4,5}

Using method:

print(a.union(b))

2. Intersection (`&`)

Returns common items.

a = {1,2,3}

b = {2,3,4}

print(a & b)

Output:

{2,3}

Using method:

print(a.intersection(b))

3. Difference (`-`)

Returns items that exist in the first set but not the second.

a = {1,2,3}

b = {2,3,4}

print(a - b)

Output:

{1}

4. Symmetric Difference (`^`)

Returns items that are different in both sets.

a = {1,2,3}

b = {2,3,4}

print(a ^ b)

Output:

{1,4}

Set Comparison

Subset

Checks if one set is inside another.

a = {1,2}

b = {1,2,3,4}

print(a.issubset(b))

Output:

True

Superset

Checks if a set contains another set.

print(b.issuperset(a))

Output:

True

Frozen Set

A frozenset is an immutable set.

Example:

numbers = frozenset([1,2,3])

print(numbers)

You cannot add or remove items.

Removing Duplicates Using Set

A common use of sets:

numbers = [1,2,2,3,4,4,5]

unique = set(numbers)

print(unique)

Output:

{1,2,3,4,5}

Convert back to list:

numbers = list(set(numbers))

Practical Programs

1. Find Common Friends

john = {"Alice","Bob","Tom"}

mary = {"Bob","Tom","Sam"}

common = john & mary

print(common)

Output:

{'Bob','Tom'}

2. Remove Duplicate Words

sentence = "python is easy and python is powerful"

words = sentence.split()

unique_words = set(words)

print(unique_words)

3. Find Missing Numbers

numbers = {1,2,4,5}

all_numbers = {1,2,3,4,5}

missing = all_numbers - numbers

print(missing)

Output:

{3}

Set vs List vs Tuple

Feature	List	Tuple	Set
Ordered	Yes	Yes	No
Changeable	Yes	No	Yes
Duplicate	Allowed	Allowed	Not Allowed
Indexing	Yes	Yes	No
Syntax	`[ ]`	`( )`	`{ }`

Practice Exercises

Create a set of 10 numbers.
Add and remove items from a set.
Remove duplicates from a list using a set.
Find common elements between two lists.
Find unique words in a sentence.
Find union, intersection, and difference of two sets.

Chapter 13: Python Dictionaries

What is a Dictionary?

A dictionary is a collection used to store data in key-value pairs.

A dictionary works like a real dictionary:

Word → Meaning
Key → Value

Example:

student = {
    "name": "John",
    "age": 20,
    "course": "Python"
}

print(student)

Output:

{'name': 'John', 'age': 20, 'course': 'Python'}

Dictionary Features

A dictionary is:

✅ Ordered (Python 3.7+ maintains insertion order)
✅ Changeable (mutable)
✅ Does not allow duplicate keys
✅ Stores data as key-value pairs

Creating a Dictionary

Example 1: Empty Dictionary

data = {}

print(data)

Output:

{}

Example 2: Student Dictionary

student = {
    "name": "Alice",
    "age": 22,
    "marks": 95
}

print(student)

Example 3: Different Data Types

person = {
    "name": "John",
    "age": 25,
    "height": 5.8,
    "student": True
}

print(person)

Accessing Dictionary Values

Using Key

student = {
    "name": "John",
    "age": 20
}

print(student["name"])

Output:

John

Using `get()`

get() is safer because it does not give an error if the key does not exist.

student = {
    "name": "John"
}

print(student.get("name"))

Output:

John

Missing key:

print(student.get("age"))

Output:

None

Difference Between `[]` and `get()`

Using brackets:

student["age"]

If the key does not exist:

KeyError

Using get:

student.get("age")

Returns:

None

Adding New Items

Example:

student = {
    "name": "John"
}

student["age"] = 20

print(student)

Output:

{'name':'John','age':20}

Updating Dictionary Values

student = {
    "name": "John",
    "age": 20
}

student["age"] = 21

print(student)

Output:

{'name':'John','age':21}

Adding Multiple Values

Using update():

student = {
    "name":"John"
}

student.update({
    "age":20,
    "city":"Delhi"
})

print(student)

Output:

{'name':'John','age':20,'city':'Delhi'}

Removing Dictionary Items

1. `pop()`

Removes a specific key.

student = {
    "name":"John",
    "age":20
}

student.pop("age")

print(student)

Output:

{'name':'John'}

2. `popitem()`

Removes the last inserted item.

student = {
    "name":"John",
    "age":20
}

student.popitem()

print(student)

Output:

{'name':'John'}

3. `del`

Deletes a key.

student = {
    "name":"John",
    "age":20
}

del student["age"]

print(student)

4. `clear()`

Removes everything.

student = {
    "name":"John"
}

student.clear()

print(student)

Output:

{}

Dictionary Methods

1. `keys()`

Returns all keys.

student = {
    "name":"John",
    "age":20
}

print(student.keys())

Output:

dict_keys(['name','age'])

2. `values()`

Returns all values.

print(student.values())

Output:

dict_values(['John',20])

3. `items()`

Returns key-value pairs.

print(student.items())

Output:

dict_items([('name','John'),('age',20)])

Loop Through Dictionary

Loop Keys

student = {
    "name":"John",
    "age":20,
    "city":"Delhi"
}

for key in student:
    print(key)

Output:

name
age
city

Loop Values

for value in student.values():
    print(value)

Output:

John
20
Delhi

Loop Keys and Values

Using items():

for key,value in student.items():
    print(key, ":", value)

Output:

name : John
age : 20
city : Delhi

Checking Key Exists

Use in:

student = {
    "name":"John"
}

if "name" in student:
    print("Found")

Output:

Found

Dictionary Length

student = {
    "name":"John",
    "age":20
}

print(len(student))

Output:

Nested Dictionary

A dictionary inside another dictionary.

Example:

students = {

    "student1": {
        "name":"John",
        "age":20
    },

    "student2": {
        "name":"Alice",
        "age":22
    }

}

print(students["student1"]["name"])

Output:

John

Dictionary with List

student = {
    "name":"John",
    "marks":[90,85,95]
}

print(student["marks"])

Output:

[90,85,95]

Access:

print(student["marks"][0])

Output:

Dictionary Comprehension

Similar to list comprehension.

Syntax:

dictionary = {key:value for item in iterable}

Example:

numbers = range(1,6)

squares = {
    x:x*x
    for x in numbers
}

print(squares)

Output:

{
1:1,
2:4,
3:9,
4:16,
5:25
}

Practical Programs

1. Count Word Frequency

sentence = "python is easy python is powerful"

words = sentence.split()

count = {}

for word in words:

    if word in count:
        count[word] += 1

    else:
        count[word] = 1

print(count)

Output:

{
'python':2,
'is':2,
'easy':1,
'powerful':1
}

2. Student Database

students = {
    101:"John",
    102:"Alice",
    103:"Bob"
}

roll = int(input("Enter roll number: "))

print(students.get(roll,"Not Found"))

3. Store Product Information

product = {
    "name":"Laptop",
    "price":50000,
    "brand":"Dell"
}

for key,value in product.items():
    print(key,value)

Dictionary vs List vs Tuple vs Set

Feature	List	Tuple	Set	Dictionary
Stores	Values	Values	Values	Key-Value
Ordered	Yes	Yes	No	Yes
Mutable	Yes	No	Yes	Yes
Duplicate	Yes	Yes	No	Keys No
Index	Yes	Yes	No	Key

Practice Exercises

Create a dictionary of your personal information.
Add and update dictionary values.
Create a phone book using a dictionary.
Count characters in a string using a dictionary.
Create a student marks database.
Create a shopping cart using a dictionary.

Chapter 14: Python Functions

What is a Function?

A function is a reusable block of code that performs a specific task.

Instead of writing the same code many times, we create a function once and use it whenever needed.

Example without function:

print("Hello John")
print("Hello Alice")
print("Hello Bob")

Using a function:

def greet(name):
    print("Hello", name)

greet("John")
greet("Alice")
greet("Bob")

Output:

Hello John
Hello Alice
Hello Bob

Why Use Functions?

Functions help to:

Reduce repeated code
Make programs easier to understand
Make debugging easier
Organize large programs
Reuse code

Creating a Function

Syntax

def function_name():
    # code

Example:

def hello():
    print("Hello Python")

hello()

Output:

Hello Python

Function Calling

Creating a function does not run it.

You must call it.

Example:

def welcome():
    print("Welcome")

welcome()

Output:

Welcome

Function with Parameters

Parameters allow you to send data into a function.

Example:

def greet(name):
    print("Hello", name)

greet("John")

Output:

Hello John

Here:

name → parameter
"John" → argument

Multiple Parameters

A function can have multiple parameters.

def add(a, b):
    print(a + b)

add(10, 20)

Output:

Return Statement

The return statement sends a value back from a function.

Example:

def add(a, b):
    return a + b

result = add(5, 3)

print(result)

Output:

Difference Between `print()` and `return`

Using print:

def add(a,b):
    print(a+b)

It only displays the result.

Using return:

def add(a,b):
    return a+b

The value can be stored and reused.

Example:

result = add(5,5)

new_value = result * 2

print(new_value)

Output:

Default Arguments

A default value is used when no argument is provided.

Example:

def greet(name="Guest"):
    print("Hello", name)

greet()
greet("John")

Output:

Hello Guest
Hello John

Keyword Arguments

Arguments can be passed by parameter name.

Example:

def student(name, age):
    print(name, age)

student(age=20, name="Alice")

Output:

Alice 20

Positional Arguments

Arguments are matched by position.

def student(name, age):
    print(name, age)

student("John", 25)

Output:

John 25

Variable Length Arguments

Sometimes we don’t know how many arguments will be passed.

Python provides:

*args
**kwargs

`*args`

Used for multiple positional arguments.

Example:

def add(*numbers):

    total = 0

    for num in numbers:
        total += num

    return total


print(add(1,2,3,4))

Output:

`**kwargs`

Used for multiple keyword arguments.

Example:

def details(**info):

    print(info)


details(name="John", age=25, city="Delhi")

Output:

{'name':'John','age':25,'city':'Delhi'}

Passing List to Function

def show(numbers):

    for n in numbers:
        print(n)


values = [10,20,30]

show(values)

Output:

10
20
30

Passing Dictionary to Function

def display(data):

    for key,value in data.items():
        print(key,value)


student = {
    "name":"John",
    "age":20
}

display(student)

Function Returning Multiple Values

Python can return multiple values.

Example:

def calculate(a,b):

    add = a+b
    multiply = a*b

    return add, multiply


x,y = calculate(5,3)

print(x)
print(y)

Output:

8
15

Local and Global Variables

Local Variable

Created inside a function.

def test():

    x = 10

    print(x)


test()

x exists only inside the function.

Global Variable

Created outside a function.

x = 100

def test():

    print(x)


test()

Output:

Global Keyword

Used to modify a global variable.

x = 10

def change():

    global x

    x = 50


change()

print(x)

Output:

Lambda Functions

A lambda function is a small anonymous function.

Syntax:

lambda arguments : expression

Example:

square = lambda x: x*x

print(square(5))

Output:

Lambda with Multiple Arguments

add = lambda a,b: a+b

print(add(10,20))

Output:

Recursive Functions

A function calling itself is called recursion.

Example: Factorial

def factorial(n):

    if n == 1:
        return 1

    return n * factorial(n-1)


print(factorial(5))

Output:

Practical Programs

1. Calculator Function

def calculator(a,b,operation):

    if operation == "+":
        return a+b

    elif operation == "-":
        return a-b

    elif operation == "*":
        return a*b

    elif operation == "/":
        return a/b


print(calculator(10,5,"+"))

2. Check Even Number

def is_even(number):

    if number % 2 == 0:
        return True

    return False


print(is_even(10))

Output:

True

3. Find Maximum Number

def maximum(numbers):

    return max(numbers)


values = [10,50,20]

print(maximum(values))

Function Naming Rules

Good:

calculate_total()
student_details()

Bad:

x()
abc123()

Use meaningful names.

Summary

You learned:

✅ Creating functions
✅ Calling functions
✅ Parameters
✅ Arguments
✅ Return values
✅ Default arguments
✅ Keyword arguments
✅ *args
✅ **kwargs
✅ Lambda functions
✅ Recursive functions
✅ Local and global variables

Practice Exercises

Create a function to add two numbers.
Create a function to check whether a number is prime.
Create a function to calculate factorial.
Create a function to count vowels in a string.
Create a calculator using functions.
Create a function that accepts a list and returns the largest number.

Chapter 15: Python Modules and Packages

What is a Module?

A module is a Python file that contains code such as:

Functions
Variables
Classes
Statements

Modules help us organize code and reuse it in different programs.

Example:

A file:

math_tools.py

contains:

def add(a, b):
    return a + b

Another Python file can use this function.

Why Use Modules?

Modules help to:

Reuse code
Keep programs organized
Reduce duplicate code
Make large projects easier to manage

Creating Your Own Module

Create a file:

calculator.py

Code:

def add(a, b):
    return a + b


def subtract(a, b):
    return a - b

Now create another file:

main.py

Import the module:

import calculator

result = calculator.add(10, 5)

print(result)

Output:

Import Statement

The import keyword is used to load modules.

Syntax:

import module_name

Example:

import math

print(math.sqrt(25))

Output:

5.0

Import Specific Functions

Instead of importing the whole module:

from math import sqrt

print(sqrt(16))

Output:

4.0

Import Multiple Functions

from math import sqrt, factorial

print(sqrt(9))
print(factorial(5))

Output:

3.0
120

Import Everything

from math import *

Example:

print(sqrt(25))
print(pow(2,3))

Output:

5.0
8.0

However, importing everything is not recommended in large projects because it can create naming conflicts.

Module Aliasing

We can give a module a shorter name.

Example:

import math as m

print(m.sqrt(49))

Output:

7.0

Built-in Python Modules

Python comes with many ready-made modules.

Some important ones:

Module	Purpose
`math`	Mathematical operations
`random`	Random numbers
`datetime`	Date and time
`os`	Operating system tasks
`sys`	Python system information
`json`	JSON data handling
`re`	Regular expressions
`statistics`	Statistical calculations

1. Math Module

Import:

import math

Square Root

print(math.sqrt(64))

Output:

8.0

Power

print(math.pow(2,3))

Output:

8.0

Ceiling

Rounds upward.

print(math.ceil(4.2))

Output:

Floor

Rounds downward.

print(math.floor(4.9))

Output:

2. Random Module

Used for random values.

import random

Random Number

number = random.randint(1,10)

print(number)

Possible output:

Random Choice

colors = ["Red","Blue","Green"]

print(random.choice(colors))

Possible output:

Blue

3. DateTime Module

Used for date and time.

import datetime

Current Date and Time

now = datetime.datetime.now()

print(now)

Example output:

2026-07-20 12:30:15

Current Date

today = datetime.date.today()

print(today)

4. OS Module

Used for operating system operations.

import os

Current Directory

print(os.getcwd())

List Files

print(os.listdir())

5. JSON Module

JSON is used for storing and transferring data.

Import:

import json

Convert Python Dictionary to JSON

import json

student = {
    "name":"John",
    "age":20
}

data = json.dumps(student)

print(data)

Output:

{"name":"John","age":20}

Convert JSON to Python Dictionary

import json

data = '{"name":"John","age":20}'

student = json.loads(data)

print(student["name"])

Output:

John

Creating a Package

What is a Package?

A package is a collection of modules stored inside a folder.

Example structure:

my_package/
│
├── __init__.py
├── calculator.py
└── converter.py

A package helps organize large applications.

Installing External Packages

Python uses pip to install packages.

Syntax:

pip install package_name

Example:

pip install requests

Using External Package

Example:

import requests

response = requests.get("https://example.com")

print(response.status_code)

Useful Python Packages

Package	Use
`requests`	Working with APIs
`numpy`	Numerical computing
`pandas`	Data analysis
`matplotlib`	Charts and graphs
`flask`	Web development
`django`	Large web applications
`openpyxl`	Excel files
`beautifulsoup4`	Web scraping

Checking Installed Packages

Command:

pip list

Shows installed packages.

Creating a Simple Project Using Modules

Project:

BankApp/

│
├── main.py
├── account.py
└── customer.py

account.py:

def deposit(amount):
    return amount

main.py:

import account

money = account.deposit(500)

print(money)

Output:

Special Variable: `name`

Every Python file has a special variable:

__name__

Example:

print(__name__)

If the file is executed directly:

Output:

__main__

Using `if name == "main"`

Example:

def hello():
    print("Hello")


if __name__ == "__main__":
    hello()

This code runs only when the file is executed directly.

Practical Programs

1. Random Password Generator

import random

characters = "abcdefghijklmnopqrstuvwxyz123456789"

password = ""

for i in range(8):
    password += random.choice(characters)

print(password)

2. Date Calculator

import datetime

today = datetime.date.today()

print("Today:", today)

3. Mathematical Calculator

import math

number = int(input("Enter number: "))

print("Square root:", math.sqrt(number))

Practice Exercises

Create your own module with addition and subtraction functions.
Use the random module to create a dice simulator.
Use datetime to print your age.
Create a package with two modules.
Install and use the requests package.
Create a simple Python project using multiple files.

Chapter 16: Python File Handling

What is File Handling?

File handling allows Python programs to store and manage data permanently.

Normally, variables store data temporarily. When the program stops, the data is lost.

Files allow us to:

Save data permanently
Read stored information
Update existing data
Create reports
Store user information

Types of Files

Python mainly works with two types of files:

1. Text Files

Store normal text.

Examples:

.txt
.csv
.json

Example:

student.txt

2. Binary Files

Store binary data.

Examples:

.jpg
.mp3
.exe
.pdf

Opening a File

Python uses the open() function.

Syntax:

file = open("filename", "mode")

Example:

file = open("data.txt", "r")

File Modes

Mode	Purpose
`r`	Read file
`w`	Write new data
`a`	Add data (append)
`x`	Create new file
`b`	Binary mode
`t`	Text mode

1. Reading a File

Create a file:

message.txt

Content:

Hello Python
Learning File Handling

Using `read()`

file = open("message.txt", "r")

data = file.read()

print(data)

file.close()

Output:

Hello Python
Learning File Handling

Closing a File

Always close files after use.

file.close()

Why?

Saves resources
Prevents data corruption
Releases memory

Using `with` Statement

The recommended method:

with open("message.txt","r") as file:

    data = file.read()

    print(data)

The file closes automatically.

Reading Specific Characters

Example:

with open("message.txt","r") as file:

    data = file.read(5)

    print(data)

Output:

Hello

Reading Lines

`readline()`

Reads one line.

with open("message.txt","r") as file:

    line = file.readline()

    print(line)

`readlines()`

Reads all lines into a list.

with open("message.txt","r") as file:

    lines = file.readlines()

    print(lines)

Output:

['Hello Python\n','Learning File Handling']

Loop Through File Lines

with open("message.txt","r") as file:

    for line in file:
        print(line)

2. Writing to a File

Mode:

Example:

file = open("data.txt","w")

file.write("Hello Python")

file.close()

Creates:

data.txt

Content:

Hello Python

Important: Write Mode Deletes Old Data

Example:

Old file:

Hello

Code:

open("data.txt","w")

New content replaces old content.

Writing Multiple Lines

with open("data.txt","w") as file:

    file.write("Line 1\n")
    file.write("Line 2\n")
    file.write("Line 3")

File:

Line 1
Line 2
Line 3

3. Appending Data

Mode:

Adds data without deleting old content.

Example:

Existing file:

Hello

Code:

with open("data.txt","a") as file:

    file.write("\nPython")

Result:

Hello
Python

4. Creating a New File

Mode:

Example:

file = open("newfile.txt","x")

file.close()

Creates a new file.

If file already exists:

FileExistsError

File Properties

File Name

file.name

Example:

with open("data.txt") as file:

    print(file.name)

Output:

data.txt

File Mode

file.mode

Example:

with open("data.txt","r") as file:

    print(file.mode)

Output:

Check File Closed

file.closed

File Cursor Position

Python keeps track of where it is reading.

`tell()`

Returns current position.

with open("data.txt","r") as file:

    print(file.tell())

`seek()`

Moves cursor position.

Example:

with open("data.txt","r") as file:

    file.seek(5)

    print(file.read())

Working with CSV Files

CSV means:

Comma Separated Values

Example:

name,age,city
John,20,Delhi
Alice,22,Mumbai

Python provides:

csv module

Writing CSV File

import csv

with open("students.csv","w") as file:

    writer = csv.writer(file)

    writer.writerow(["Name","Age"])

    writer.writerow(["John",20])

Reading CSV File

import csv

with open("students.csv","r") as file:

    reader = csv.reader(file)

    for row in reader:
        print(row)

Working with JSON Files

JSON stores structured data.

Example:

{
"name":"John",
"age":20
}

Writing JSON

import json

student = {
    "name":"John",
    "age":20
}


with open("student.json","w") as file:

    json.dump(student,file)

Reading JSON

import json


with open("student.json","r") as file:

    data = json.load(file)

    print(data)

Output:

{'name':'John','age':20}

Exception Handling with Files

Sometimes files may not exist.

Example:

try:

    file = open("abc.txt","r")

    print(file.read())

except FileNotFoundError:

    print("File not found")

Output:

File not found

Practical Projects

1. Notes Application

note = input("Write your note: ")

with open("notes.txt","a") as file:

    file.write(note+"\n")

print("Saved")

2. Student Record System

name = input("Name: ")
marks = input("Marks: ")


with open("students.txt","a") as file:

    file.write(name+" "+marks+"\n")

3. Count Words in a File

with open("data.txt","r") as file:

    text = file.read()

    words = text.split()

    print(len(words))

Best Practices

Always prefer:

with open("file.txt","r") as file:

Instead of:

file=open("file.txt","r")

Because it automatically closes the file.

Practice Exercises

Create a text file and write your name.
Read a file and count characters.
Create a program to store student records.
Copy contents from one file to another.
Create a CSV file for employee data.
Store and read JSON user information.

Chapter 17: Python Exception Handling

What is an Exception?

An exception is an error that occurs while a program is running.

When an exception occurs, Python stops the program unless we handle it.

Example:

number = 10 / 0

print(number)

Output:

ZeroDivisionError

The program crashes because division by zero is not possible.

Why Use Exception Handling?

Exception handling helps to:

Prevent program crashes
Show meaningful error messages
Handle unexpected situations
Make programs more reliable

Types of Errors in Python

1. Syntax Error

Occurs when Python grammar is incorrect.

Example:

print("Hello"

Output:

SyntaxError

2. Runtime Error

Occurs while the program is running.

Example:

x = 10 / 0

Output:

ZeroDivisionError

3. Logical Error

Program runs but gives the wrong result.

Example:

price = 100
discount = 20

total = price + discount

print(total)

The code runs, but the calculation is wrong.

Common Python Exceptions

Exception	Meaning
`ZeroDivisionError`	Division by zero
`ValueError`	Wrong value type
`TypeError`	Wrong data type
`IndexError`	Invalid index
`KeyError`	Missing dictionary key
`FileNotFoundError`	File does not exist
`NameError`	Variable not defined

try and except

The basic structure:

try:
    # risky code

except:
    # error handling

Example:

try:

    x = 10 / 0

except:

    print("Something went wrong")

Output:

Something went wrong

Handling Specific Exceptions

It is better to catch specific errors.

Example:

try:

    number = int(input("Enter number: "))

    print(10 / number)


except ZeroDivisionError:

    print("Cannot divide by zero")


except ValueError:

    print("Please enter a number")

Multiple Exceptions

Example:

try:

    a = int(input("Enter: "))

    b = int(input("Enter: "))

    print(a/b)


except (ValueError, ZeroDivisionError):

    print("Invalid input")

Using Exception Object

You can store the error message.

Example:

try:

    x = 10 / 0


except Exception as error:

    print(error)

Output:

division by zero

try with else

The else block runs when no exception occurs.

Syntax:

try:

    code

except:

    error

else:

    success

Example:

try:

    number = int(input("Enter number: "))


except ValueError:

    print("Invalid number")


else:

    print("You entered:", number)

try with finally

The finally block always runs.

Example:

try:

    file = open("data.txt","r")

    print(file.read())


except:

    print("File error")


finally:

    print("Program finished")

Output:

Program finished

File Handling with Exception

Example:

try:

    with open("abc.txt","r") as file:

        print(file.read())


except FileNotFoundError:

    print("File does not exist")

Raising Exceptions

Sometimes we create our own errors using raise.

Syntax:

raise ExceptionType("message")

Example:

age = -5

if age < 0:

    raise ValueError("Age cannot be negative")

Output:

ValueError: Age cannot be negative

Custom Exceptions

We can create our own exception classes.

Example:

class AgeError(Exception):
    pass


age = 10


if age < 18:

    raise AgeError("Not eligible")

User Input Exception Handling

Example:

while True:

    try:

        age = int(input("Enter age: "))

        break


    except ValueError:

        print("Enter numbers only")

The program keeps asking until valid input is entered.

Exception Handling in Functions

Example:

def divide(a,b):

    try:

        return a/b


    except ZeroDivisionError:

        return "Cannot divide by zero"


print(divide(10,0))

Output:

Cannot divide by zero

Exception Handling Flow

Example:

try:
    risky code

except:
    handle error

else:
    no error

finally:
    always execute

Flow:

Start
  |
  v
Try Block
  |
  +---- Error? ---- Yes ----> Except Block
  |
  No
  |
  v
Else Block
  |
  v
Finally Block
  |
  v
End

Real-World Examples

1. Banking System

balance = 5000

try:

    withdraw = int(input("Withdraw amount: "))

    if withdraw > balance:

        raise ValueError("Insufficient balance")

    balance -= withdraw

    print("Remaining:", balance)


except ValueError as e:

    print(e)

2. Login System

password = "python123"

try:

    user_password = input("Password: ")

    if user_password != password:

        raise Exception("Wrong password")


    print("Login successful")


except Exception as e:

    print(e)

Best Practices

Use specific exceptions

Good:

except ValueError:

Avoid:

except:

because it hides all errors.

Keep try block small

Good:

try:
    number = int(value)

Avoid putting the whole program inside try.

Practice Exercises

Create a calculator with exception handling.
Handle invalid user input.
Create a program that handles missing files.
Create a custom exception for password errors.
Create a bank withdrawal system with exceptions.
Handle division errors.

Chapter 19: Advanced Object-Oriented Programming (OOP)

In the previous chapter, we learned the basics of classes and objects.

Now we will learn advanced OOP concepts used in professional Python projects.

1. Magic Methods (Dunder Methods)

What are Magic Methods?

Magic methods are special methods in Python that start and end with double underscores (__).

Example:

__init__()
__str__()
__len__()
__add__()

They allow objects to work with Python built-in functions and operators.

`init()` Constructor

Called automatically when an object is created.

Example:

class Student:

    def __init__(self, name):
        self.name = name


student = Student("John")

`str()` Method

Controls how an object appears when printed.

Without __str__:

class Student:

    def __init__(self,name):
        self.name = name


student = Student("John")

print(student)

Output:

<__main__.Student object>

Using __str__():

class Student:

    def __init__(self,name):
        self.name = name


    def __str__(self):
        return self.name


student = Student("John")

print(student)

Output:

John

`len()` Method

Controls the behavior of len().

Example:

class Team:

    def __init__(self,players):
        self.players = players


    def __len__(self):
        return len(self.players)



team = Team(["John","Alice","Bob"])

print(len(team))

Output:

`del()` Destructor

Runs when an object is deleted.

Example:

class Student:

    def __del__(self):

        print("Object deleted")


student = Student()

del student

Output:

Object deleted

2. Operator Overloading

Python allows changing how operators work with objects.

Example:

Normally:

5 + 10

Output:

We can define how + works for our objects.

Using `add()`

Example:

class Number:

    def __init__(self,value):

        self.value = value


    def __add__(self,other):

        return self.value + other.value



a = Number(10)

b = Number(20)


print(a+b)

Output:

Common Operator Methods

Operator	Method
`+`	`__add__()`
`-`	`__sub__()`
`*`	`__mul__()`
`/`	`__truediv__()`
`==`	`__eq__()`
`<`	`__lt__()`
`>`	`__gt__()`

3. Property Decorator

Problem

Suppose we have:

class Person:

    def __init__(self,age):

        self.age = age

A user can set:

person.age = -100

This is incorrect.

Solution: `@property`

Example:

class Person:

    def __init__(self,age):

        self._age = age


    @property
    def age(self):

        return self._age


    @age.setter
    def age(self,value):

        if value < 0:

            print("Age cannot be negative")

        else:

            self._age = value



person = Person(20)

person.age = 25

print(person.age)

Output:

4. Multiple Inheritance

A class can inherit from multiple classes.

Example:

class Father:

    def skill1(self):

        print("Driving")


class Mother:

    def skill2(self):

        print("Cooking")


class Child(Father,Mother):

    pass



child = Child()

child.skill1()

child.skill2()

Output:

Driving
Cooking

5. Method Resolution Order (MRO)

When multiple classes have the same method, Python decides which one to use.

Example:

class A:

    def show(self):

        print("A")


class B(A):

    def show(self):

        print("B")


obj = B()

obj.show()

Output:

Checking MRO:

print(B.mro())

Output:

[B, A, object]

Python searches from left to right.

6. Calling Parent Class Methods

Use:

super()

Example:

class Animal:

    def sound(self):

        print("Animal sound")


class Dog(Animal):

    def sound(self):

        super().sound()

        print("Bark")


dog = Dog()

dog.sound()

Output:

Animal sound
Bark

7. Abstract Classes

Abstract classes provide a blueprint for other classes.

Python uses:

from abc import ABC, abstractmethod

Example:

from abc import ABC, abstractmethod


class Shape(ABC):

    @abstractmethod
    def area(self):

        pass

Child class must implement it:

class Square(Shape):

    def area(self):

        print("Area of square")

8. Dataclasses

Dataclasses reduce repetitive code.

Normal class:

class Student:

    def __init__(self,name,age):

        self.name=name
        self.age=age

Using dataclass:

from dataclasses import dataclass


@dataclass
class Student:

    name:str
    age:int



student = Student("John",20)

print(student)

Output:

Student(name='John', age=20)

9. Class Composition

Composition means one class contains another class.

Example:

A car has an engine.

class Engine:

    def start(self):

        print("Engine started")


class Car:

    def __init__(self):

        self.engine = Engine()


    def drive(self):

        self.engine.start()

        print("Car moving")



car = Car()

car.drive()

Output:

Engine started
Car moving

10. Association

Objects can work together.

Example:

class Teacher:

    def teach(self):

        print("Teaching")


class Student:

    def learn(self,teacher):

        teacher.teach()



teacher = Teacher()

student = Student()

student.learn(teacher)

Real-World OOP Example: E-Commerce System

class Product:

    def __init__(self,name,price):

        self.name=name
        self.price=price



class Cart:

    def __init__(self):

        self.products=[]


    def add_product(self,product):

        self.products.append(product)


    def total(self):

        return sum(
            p.price for p in self.products
        )



p1 = Product("Laptop",50000)

p2 = Product("Mouse",1000)


cart = Cart()

cart.add_product(p1)

cart.add_product(p2)


print(cart.total())

Output:

Practice Exercises

Create a class with __str__() method.
Create a BankAccount class with property validation.
Create operator overloading for adding two objects.
Create multiple inheritance example.
Create an abstract Shape class.
Create an e-commerce system using OOP.

Chapter 20: Python Iterators and Generators

What is Iteration?

Iteration means going through items one by one.

Examples:

Reading each character of a string
Accessing each item in a list
Processing rows in a file

Example:

id="q8w4mz"
numbers = [10,20,30]

for num in numbers:
    print(num)

Output:

10
20
30

The for loop internally uses iterators.

Iterable Objects

An iterable is an object that can return its items one at a time.

Examples:

id="x5m8pq"
list
tuple
string
dictionary
set
range

Example:

id="k9v2za"
name = "Python"

for letter in name:
    print(letter)

Output:

P
y
t
h
o
n

Checking if an Object is Iterable

Use:

id="m4q8vx"
from collections.abc import Iterable

print(isinstance([1,2,3], Iterable))

Output:

True

What is an Iterator?

An iterator is an object that remembers its current position while going through data.

An iterator has two methods:

__iter__()
__next__()

Creating an Iterator

Using iter():

id="r7m3qx"
numbers = [10,20,30]

iterator = iter(numbers)

print(iterator)

Output:

<list_iterator>

Using `next()`

next() gets the next item.

Example:

id="p9x4mv"
numbers = [10,20,30]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

Output:

10
20
30

StopIteration Exception

When no items remain, Python raises:

StopIteration

Example:

id="z8m5qx"
numbers = [1,2]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

Output:

1
2
StopIteration

How for Loop Works Internally

When we write:

id="f6q2mx"
for item in [1,2,3]:
    print(item)

Python internally does:

id="n7v4pz"
iterator = iter([1,2,3])

while True:

    try:
        item = next(iterator)
        print(item)

    except StopIteration:
        break

Creating Your Own Iterator

Example:

id="u5m9qx"
class Count:

    def __init__(self,max):

        self.max = max
        self.current = 1


    def __iter__(self):

        return self


    def __next__(self):

        if self.current <= self.max:

            value = self.current

            self.current += 1

            return value

        else:

            raise StopIteration



counter = Count(5)


for number in counter:

    print(number)

Output:

What are Generators?

A generator is a special type of function that produces values one at a time.

Generators use:

yield

instead of:

return

Normal Function vs Generator

Normal Function

id="y8p3mx"
def numbers():

    return [1,2,3,4,5]

It creates the entire list in memory.

Generator

id="q5m7vx"
def numbers():

    yield 1
    yield 2
    yield 3

Values are created only when needed.

Creating a Generator

Example:

id="a9q4mz"
def count():

    yield 1
    yield 2
    yield 3


generator = count()


print(next(generator))
print(next(generator))
print(next(generator))

Output:

1
2
3

Generator with Loop

id="w6m2px"
def count():

    yield 1
    yield 2
    yield 3


for value in count():

    print(value)

Output:

1
2
3

Generator with Range

Example:

id="d7q3mx"
def numbers(n):

    for i in range(n):

        yield i


for x in numbers(5):

    print(x)

Output:

Why Use Generators?

Generators are useful because they:

Use less memory
Work with large data
Improve performance
Produce values when needed

Example: Large Data Processing

Without generator:

id="v4m8qx"
numbers = [x for x in range(1000000)]

Creates one million items immediately.

With generator:

id="k8q2mz"
numbers = (x for x in range(1000000))

Creates values only when requested.

Generator Expression

Similar to list comprehension.

List Comprehension

id="p3x7mq"
numbers = [x*x for x in range(5)]

print(numbers)

Output:

[0,1,4,9,16]

Generator Expression

id="m9v4qx"
numbers = (x*x for x in range(5))

print(next(numbers))

Output:

List vs Generator

Feature	List	Generator
Memory	More	Less
Speed	Faster for small data	Better for large data
Stores all values	Yes	No
Uses	`[ ]`	`( )`
Keyword	None	`yield`

Real-World Examples

1. Reading Large Files

Generator:

id="t8m3vz"
def read_file(filename):

    with open(filename) as file:

        for line in file:

            yield line



for line in read_file("data.txt"):

    print(line)

It reads one line at a time.

2. Infinite Generator

Generators can create unlimited data.

Example:

id="q4x7mn"
def infinite_numbers():

    number = 1

    while True:

        yield number

        number += 1



gen = infinite_numbers()


print(next(gen))
print(next(gen))
print(next(gen))

Output:

1
2
3

3. Fibonacci Generator

id="h5m8qx"
def fibonacci(limit):

    a,b = 0,1

    for i in range(limit):

        yield a

        a,b = b,a+b



for number in fibonacci(10):

    print(number)

Output:

Generator Methods

`send()`

Sends a value into a generator.

Example:

id="z7q3mv"
def test():

    value = yield

    print(value)


g = test()

next(g)

g.send("Hello")

Output:

Hello

Generator Pipeline

Generators can be connected together.

Example:

id="b8m4qx"
def numbers():

    for i in range(10):

        yield i


def squares(data):

    for x in data:

        yield x*x



result = squares(numbers())


for value in result:

    print(value)

Practice Exercises

Create an iterator that counts from 1 to 10.
Create a generator for even numbers.
Create a Fibonacci generator.
Create a generator to read a file line by line.
Convert a list comprehension into a generator expression.
Create an infinite counter generator.

Chapter 21: Python Decorators and Advanced Functions

What are Advanced Functions?

In Python, functions are treated like objects.

This means:

A function can be stored in a variable
A function can be passed as an argument
A function can return another function
A function can be modified without changing its code

This feature allows powerful concepts like:

Closures
Decorators
Functional programming

1. First-Class Functions

A programming language has first-class functions when functions can be used like normal values.

Storing Function in a Variable

Example:

def hello():

    print("Hello Python")


x = hello

x()

Output:

Hello Python

Here:

x = hello

stores the function inside a variable.

Passing Function as an Argument

Example:

def greet():

    return "Hello"


def display(function):

    print(function())


display(greet)

Output:

Hello

A function can be passed into another function.

Returning a Function

A function can return another function.

Example:

def outer():

    def inner():

        print("Inside inner function")


    return inner


result = outer()

result()

Output:

Inside inner function

2. Nested Functions

A function inside another function is called a nested function.

Example:

def outer():

    print("Outer function")


    def inner():

        print("Inner function")


    inner()


outer()

Output:

Outer function
Inner function

Why Use Nested Functions?

They help to:

Hide helper functions
Organize complex logic
Create closures
Build decorators

3. Closures

What is a Closure?

A closure is a function that remembers variables from its outer function even after the outer function has finished.

Example:

def multiplier(x):

    def calculate(y):

        return x * y


    return calculate


double = multiplier(2)

print(double(5))

Output:

The inner function remembers:

x = 2

Closure Example

def counter():

    count = 0


    def increase():

        nonlocal count

        count += 1

        return count


    return increase



c = counter()

print(c())
print(c())
print(c())

Output:

1
2
3

4. What is a Decorator?

A decorator is a function that modifies another function without changing its original code.

Syntax:

@decorator
def function():
    pass

Simple Decorator

Example:

def decorator(func):

    def wrapper():

        print("Before function")

        func()

        print("After function")


    return wrapper



@decorator
def hello():

    print("Hello")


hello()

Output:

Before function
Hello
After function

How Decorator Works Internally

This:

@decorator
def hello():

is the same as:

hello = decorator(hello)

Decorator with Arguments

Problem:

A normal wrapper cannot handle parameters.

Example solution:

def decorator(func):

    def wrapper(*args, **kwargs):

        print("Starting")

        result = func(*args, **kwargs)

        print("Finished")

        return result


    return wrapper



@decorator
def add(a,b):

    return a+b



print(add(5,3))

Output:

Starting
Finished
8

Multiple Decorators

You can use more than one decorator.

Example:

def first(func):

    def wrapper():

        print("First")

        func()

    return wrapper



def second(func):

    def wrapper():

        print("Second")

        func()

    return wrapper



@first
@second
def hello():

    print("Hello")


hello()

Output:

First
Second
Hello

5. Built-in Decorators

Python provides built-in decorators:

@staticmethod
@classmethod
@property

@staticmethod

Used for methods that do not need object data.

Example:

class Calculator:

    @staticmethod
    def add(a,b):

        return a+b



print(Calculator.add(5,10))

Output:

@classmethod

Works with class-level data.

Example:

class Student:

    school = "ABC"


    @classmethod
    def show_school(cls):

        print(cls.school)



Student.show_school()

Output:

ABC

@property

Allows controlled access to attributes.

Example:

class Person:

    def __init__(self,age):

        self._age = age


    @property
    def age(self):

        return self._age



p = Person(20)

print(p.age)

Output:

6. Practical Decorator Examples

1. Login Authentication

def login_required(func):

    def wrapper(user):

        if user == "admin":

            func()

        else:

            print("Access denied")


    return wrapper



@login_required
def dashboard():

    print("Welcome to dashboard")



dashboard("admin")

Output:

Welcome to dashboard

2. Measuring Execution Time

import time


def timer(func):

    def wrapper():

        start = time.time()

        func()

        end = time.time()

        print("Time:", end-start)


    return wrapper



@timer
def process():

    for i in range(1000000):

        pass



process()

3. Logging Decorator

def log(func):

    def wrapper():

        print("Function started")

        func()

        print("Function ended")


    return wrapper



@log
def work():

    print("Working")



work()

7. functools.wraps

Problem:

Decorators replace function information.

Solution:

from functools import wraps

Example:

from functools import wraps


def decorator(func):

    @wraps(func)

    def wrapper():

        return func()


    return wrapper

It preserves:

Function name
Documentation
Metadata

Functional Programming Concepts

Python supports:

Map
Filter
Reduce

map()

Applies a function to every item.

Example:

numbers = [1,2,3,4]

result = list(
    map(lambda x:x*2, numbers)
)

print(result)

Output:

[2,4,6,8]

filter()

Filters items based on condition.

Example:

numbers = [1,2,3,4,5]

result = list(
    filter(lambda x:x%2==0, numbers)
)

print(result)

Output:

[2,4]

reduce()

Combines values into one result.

Example:

from functools import reduce


numbers = [1,2,3,4]


result = reduce(
    lambda a,b:a+b,
    numbers
)


print(result)

Output:

Practice Exercises

Create a decorator that prints “Start” and “End”.
Create a decorator for checking login.
Create a closure-based counter.
Create a timer decorator.
Use map() to square numbers.
Use filter() to find even numbers.
Use reduce() to calculate total.

Chapter 22: Python Regular Expressions (Regex)

What is Regular Expression?

A Regular Expression (Regex) is a pattern used to search, match, and manipulate text.

Regex is useful for:

Finding specific text
Checking user input
Validating emails
Extracting data
Searching large documents

Python provides regex through the built-in module:

import re

Import Regex Module

import re

Basic Regex Functions

Python re module provides:

Function	Purpose
`match()`	Checks pattern at beginning
`search()`	Finds first match anywhere
`findall()`	Finds all matches
`finditer()`	Returns match objects
`sub()`	Replace text
`split()`	Split string using pattern

1. re.match()

Checks if the pattern exists at the beginning.

Example:

import re

text = "Python is powerful"

result = re.match("Python", text)

print(result)

Output:

Match object

Example:

import re

text = "I love Python"

result = re.match("Python", text)

print(result)

Output:

None

Because Python is not at the start.

2. re.search()

Searches anywhere in the string.

Example:

import re

text = "I love Python"

result = re.search("Python", text)

print(result)

Output:

Match object

3. re.findall()

Returns all matches.

Example:

import re

text = "My numbers are 10, 20, 30"

result = re.findall("\d+", text)

print(result)

Output:

['10','20','30']

4. re.finditer()

Returns detailed match information.

Example:

import re

text = "Python Java Python"

result = re.finditer("Python", text)

for match in result:

    print(match.start(), match.end())

Output:

0 6
13 19

Regex Patterns

Regex uses special characters called metacharacters.

1. Character Classes

Digits

Pattern:

\d

Matches numbers.

Example:

import re

text = "Age 25"

print(re.findall("\d", text))

Output:

['2','5']

Non-digit

Pattern:

\D

Example:

text = "abc123"

print(re.findall("\D",text))

Output:

['a','b','c']

Word Characters

Pattern:

\w

Matches:

Letters
Numbers
Underscore

Example:

text = "Python_123"

print(re.findall("\w",text))

Non-word Characters

Pattern:

\W

Spaces

Pattern:

\s

Example:

text="Hello World"

print(re.findall("\s",text))

Quantifiers

Quantifiers define how many times a pattern should appear.

Symbol	Meaning
`*`	0 or more
`+`	1 or more
`?`	0 or 1
`{n}`	Exactly n times
`{n,m}`	Between n and m

`+` Example

One or more digits:

import re

text="123 abc 45"

print(re.findall("\d+",text))

Output:

['123','45']

`{}` Example

Exactly 4 digits:

text="1234 567 9999"

print(re.findall("\d{4}",text))

Output:

['1234','9999']

Special Characters

Dot `.`

Matches any character.

Example:

text="cat cot cut"

print(re.findall("c.t",text))

Output:

['cat','cot','cut']

Caret `^`

Starts with.

Example:

text="Python programming"

print(re.match("^Python",text))

Dollar `$`

Ends with.

Example:

text="hello.com"

print(re.search("com$",text))

Character Sets `[ ]`

Matches any character inside brackets.

Example:

text="cat bat mat"

print(re.findall("[cbm]at",text))

Output:

['cat','bat','mat']

Range Matching

Example:

text="abc123"

print(re.findall("[a-z]",text))

Output:

['a','b','c']

Numbers:

print(re.findall("[0-9]", "abc123"))

Output:

['1','2','3']

Grouping `( )`

Groups patterns together.

Example:

text="2026-07-20"

pattern="\d{4}-\d{2}-\d{2}"

print(re.findall(pattern,text))

Output:

['2026-07-20']

Replace Text Using re.sub()

Example:

import re

text="I like Java"

result = re.sub("Java","Python",text)

print(result)

Output:

I like Python

Split Text Using re.split()

Example:

import re

text="apple,orange;banana"

result = re.split("[,;]",text)

print(result)

Output:

['apple','orange','banana']

Real-World Regex Examples

1. Validate Mobile Number

Example:

import re

number="9876543210"

pattern="^[0-9]{10}$"

if re.match(pattern,number):

    print("Valid")

else:

    print("Invalid")

Output:

Valid

2. Email Validation

Example:

import re

email="user@gmail.com"

pattern=r"^[\w\.-]+@[\w\.-]+\.\w+$"


if re.match(pattern,email):

    print("Valid Email")

else:

    print("Invalid")

3. Extract Phone Numbers

Text:

text="Call me at 9876543210"

Code:

numbers = re.findall(r"\d{10}", text)

print(numbers)

Output:

['9876543210']

4. Password Validation

Requirements:

Minimum 8 characters
One uppercase letter
One number

Example:

password="Python123"


pattern=r"^(?=.*[A-Z])(?=.*\d).{8,}$"


if re.match(pattern,password):

    print("Strong password")

else:

    print("Weak password")

Raw Strings in Regex

Regex uses many backslashes.

Instead of:

"\d+"

Use:

r"\d+"

Example:

import re

print(re.findall(r"\d+","Age 25"))

Output:

['25']

Practical Projects

1. Text Analyzer

Features:

Count words
Find numbers
Find emails

2. Log File Analyzer

Extract:

Dates
IP addresses
Error messages

3. Data Cleaner

Remove:

Extra spaces
Special characters
Invalid data

Practice Exercises

Find all numbers in a sentence.
Extract email addresses from text.
Validate mobile numbers.
Validate passwords using regex.
Replace all spaces with _.
Extract dates from a document.
Create a simple spam detector.

Chapter 23: Python Database Programming (SQLite)

What is a Database?

A database is a place where data is stored permanently and organized so it can be easily accessed.

Examples:

Student records
Customer information
Product details
Banking data
Employee records

Database Types

1. Relational Database (SQL)

Stores data in tables.

Examples:

SQLite
MySQL
PostgreSQL
Oracle

Structure:

Table
|
|-- Rows (Records)
|
|-- Columns (Fields)

Example:

Students Table

ID	Name	Age
1	John	20
2	Alice	22

2. NoSQL Database

Stores flexible data.

Examples:

MongoDB
Redis
Cassandra

Python Database Support

Python can connect with many databases:

Database	Python Library
SQLite	sqlite3
MySQL	mysql-connector
PostgreSQL	psycopg
MongoDB	pymongo

We start with SQLite because it is built into Python.

SQLite Database

SQLite is:

Lightweight
Serverless
File-based
Included with Python

Import:

import sqlite3

Connecting to Database

Example:

import sqlite3


connection = sqlite3.connect("school.db")

print("Database connected")

This creates:

school.db

Creating a Cursor

A cursor executes SQL commands.

Example:

cursor = connection.cursor()

Creating a Table

SQL command:

CREATE TABLE students(
id INTEGER,
name TEXT,
age INTEGER
)

Python:

import sqlite3


connection = sqlite3.connect("school.db")

cursor = connection.cursor()


cursor.execute("""
CREATE TABLE students(
id INTEGER,
name TEXT,
age INTEGER
)
""")


connection.close()

Insert Data

SQL:

INSERT INTO students VALUES(1,'John',20)

Python:

import sqlite3


connection = sqlite3.connect("school.db")

cursor = connection.cursor()


cursor.execute(
"INSERT INTO students VALUES(1,'John',20)"
)


connection.commit()

connection.close()

Why Use commit()?

commit() saves changes permanently.

Without:

connection.commit()

data may be lost.

Insert Multiple Records

Use executemany().

Example:

students = [

(1,"John",20),

(2,"Alice",22),

(3,"Bob",21)

]


cursor.executemany(
"INSERT INTO students VALUES(?,?,?)",
students
)


connection.commit()

Reading Data (SELECT)

SQL:

SELECT * FROM students

Python:

cursor.execute(
"SELECT * FROM students"
)


data = cursor.fetchall()


for row in data:

    print(row)

Output:

(1,'John',20)
(2,'Alice',22)
(3,'Bob',21)

fetchone()

Gets one record.

Example:

cursor.execute(
"SELECT * FROM students"
)


print(cursor.fetchone())

Output:

(1,'John',20)

fetchmany()

Gets limited records.

Example:

data = cursor.fetchmany(2)

print(data)

Output:

[(1,'John',20),(2,'Alice',22)]

Updating Data

SQL:

UPDATE students
SET age=21
WHERE id=1

Python:

cursor.execute("""
UPDATE students
SET age=21
WHERE id=1
""")


connection.commit()

Deleting Data

SQL:

DELETE FROM students
WHERE id=1

Python:

cursor.execute("""
DELETE FROM students
WHERE id=1
""")


connection.commit()

CRUD Operations

CRUD means:

Operation	SQL
Create	INSERT
Read	SELECT
Update	UPDATE
Delete	DELETE

These are the basic database operations.

Using User Input

Example:

import sqlite3


connection = sqlite3.connect("school.db")

cursor = connection.cursor()


name = input("Enter name: ")

age = int(input("Enter age: "))


cursor.execute(
"INSERT INTO students VALUES(NULL,?,?)",
(name,age)
)


connection.commit()

connection.close()

SQL Injection Problem

Bad practice:

name = input()

query = "SELECT * FROM students WHERE name='"+name+"'"

Attackers can modify SQL commands.

Safe Query Using Parameters

Good:

cursor.execute(
"SELECT * FROM students WHERE name=?",
(name,)
)

Always use placeholders:

Creating a Database Class

Professional approach:

import sqlite3


class Database:


    def __init__(self):

        self.connection = sqlite3.connect(
            "school.db"
        )


    def insert(self,name,age):

        cursor = self.connection.cursor()

        cursor.execute(
        "INSERT INTO students VALUES(NULL,?,?)",
        (name,age)
        )

        self.connection.commit()



db = Database()

db.insert("David",25)

Using Context Manager

Better way:

with sqlite3.connect("school.db") as connection:

    cursor = connection.cursor()

    cursor.execute(
    "SELECT * FROM students"
    )

Automatically closes connection.

Primary Key

A primary key uniquely identifies each row.

Example:

id INTEGER PRIMARY KEY

Example table:

ID    Name
1     John
2     Alice

No two IDs can be the same.

Auto Increment ID

Example:

CREATE TABLE students(

id INTEGER PRIMARY KEY AUTOINCREMENT,

name TEXT,

age INTEGER

)

Now IDs are automatically created.

Foreign Key

Used to connect tables.

Example:

Students:

student_id
name

Courses:

course_id
student_id
course_name

Relationship:

Student ---- Course

Real Project: Student Management System

Features:

Add student
View students
Update student
Delete student

Example structure:

student_system.py

Database
 |
 |-- students table
 |
 |-- CRUD functions

Practice Exercises

Create a library database.
Create an employee table.
Insert 10 records.
Search records by name.
Update employee salary.
Delete old records.
Build a student management system.

Chapter 24: Python Web Development Basics

What is Web Development?

Web development is the process of creating websites and web applications.

Examples:

Online shopping websites
Social media platforms
Banking websites
Learning platforms

Python is widely used for backend development.

How Websites Work

A website has two main parts:

1. Frontend (Client Side)

The part users see.

Technologies:

HTML → Structure
CSS → Design
JavaScript → Interaction

Example:

Button
Menu
Images
Forms

2. Backend (Server Side)

The part that runs behind the scenes.

Python handles:

Business logic
Database operations
User authentication
Data processing

Client and Server

When you open a website:

User Browser
      |
      |
   Request
      |
      v
 Web Server
      |
      |
  Response
      |
      v
User Browser

Example:

You open:

example.com/products

Browser sends a request.

Server returns the webpage.

HTTP Basics

HTTP is the communication protocol between browser and server.

Common HTTP methods:

Method	Purpose
GET	Get data
POST	Send data
PUT	Update data
DELETE	Remove data

Python Web Frameworks

A framework provides tools to build websites.

Popular Python frameworks:

Flask

Simple
Beginner friendly
Lightweight

Django

Full-featured
Used for large applications

FastAPI

Modern API development
High performance

We start with Flask.

Installing Flask

Install:

pip install flask

Check:

pip show flask

First Flask Application

Create:

app.py

Code:

from flask import Flask


app = Flask(__name__)


@app.route("/")
def home():

    return "Hello Python Web"


app.run()

Run:

python app.py

Output:

Running on http://127.0.0.1:5000

Open in browser:

http://127.0.0.1:5000

Understanding Flask Code

Import Flask

from flask import Flask

Loads Flask framework.

Create Application

app = Flask(__name__)

Creates Flask object.

Route

@app.route("/")

Connects URL with a function.

Function

def home():

Runs when user visits that URL.

Creating Multiple Pages

Example:

from flask import Flask


app = Flask(__name__)


@app.route("/")
def home():

    return "Home Page"



@app.route("/about")
def about():

    return "About Page"



app.run()

URLs:

/ 
/about

Dynamic URLs

You can create URLs with variables.

Example:

@app.route("/user/<name>")
def user(name):

    return "Hello " + name

Visit:

/user/John

Output:

Hello John

URL Parameters

Example:

@app.route("/product/<int:id>")
def product(id):

    return "Product ID: " + str(id)

Visit:

/product/10

Output:

Product ID: 10

HTML Templates

Instead of returning text, websites use HTML files.

Project:

website/

 |-- app.py

 |-- templates/

       |-- home.html

Using render_template()

Example:

from flask import Flask, render_template


app = Flask(__name__)


@app.route("/")
def home():

    return render_template("home.html")


app.run()

HTML file:

home.html

<!DOCTYPE html>

<html>

<body>

<h1>
Welcome Python
</h1>

</body>

</html>

Passing Data to HTML

Python:

@app.route("/")
def home():

    name = "John"

    return render_template(
        "home.html",
        username=name
    )

HTML:

<h1>
Hello {{ username }}
</h1>

Output:

Hello John

Jinja Template Engine

Flask uses Jinja.

Features:

Variables
Loops
Conditions

Jinja Variables

Python:

name="Alice"

HTML:

{{ name }}

Jinja If Condition

HTML:

{% if age >= 18 %}

Adult

{% else %}

Minor

{% endif %}

Jinja Loop

Python:

students=[
"John",
"Alice",
"Bob"
]

HTML:

{% for student in students %}

<p>
{{student}}
</p>

{% endfor %}

Output:

John
Alice
Bob

Handling Forms

Forms send data from user to server.

HTML:

<form method="POST">

<input name="username">

<button>
Submit
</button>

</form>

Python:

from flask import request


@app.route("/login",methods=["POST"])
def login():

    username=request.form["username"]

    return username

GET vs POST

GET

Used for retrieving data.

Example:

Search page

Data appears in URL.

POST

Used for sending sensitive data.

Example:

Login form
Registration

Data is sent in request body.

Flask with Database

Example:

User
 |
 |
Flask
 |
 |
SQLite Database

Application flow:

User submits form
Flask receives data
Database stores data
Flask returns response

Creating JSON API

API allows applications to communicate.

Example:

from flask import jsonify


@app.route("/api/user")
def user():

    data={
        "name":"John",
        "age":20
    }

    return jsonify(data)

Output:

{
"name":"John",
"age":20
}

REST API Basics

REST uses HTTP methods:

Action	Method
Create	POST
Read	GET
Update	PUT
Delete	DELETE

Simple API Example

users=[]


@app.route("/users",methods=["GET"])
def get_users():

    return jsonify(users)



@app.route("/users",methods=["POST"])
def add_user():

    users.append(
        request.json
    )

    return "Added"

Flask Project Structure

Professional structure:

myproject/

│
├── app.py
│
├── templates/
│      └── index.html
│
├── static/
│      ├── style.css
│      └── script.js
│
└── database.db

Real Projects Using Flask

1. Blog Website

Features:

User login
Posts
Comments
Database

2. Online Store

Features:

Products
Cart
Orders
Payments

3. REST API

Features:

User management
Authentication
Mobile app backend

Practice Exercises

Create a Flask hello world app.
Create home and about pages.
Create a user profile page.
Build a login form.
Create a student CRUD application.
Build a simple REST API.

Chapter 25: Python API Development and FastAPI

What is an API?

API (Application Programming Interface) allows different software applications to communicate with each other.

Examples:

Mobile app ↔ Server
Website ↔ Database
Payment system ↔ Bank system

Example:

A weather app does not create weather data itself.

It requests data from a weather API:

Mobile App
     |
     | Request
     v
Weather API Server
     |
     | Response
     v
Mobile App

What is REST API?

REST means:

Representational State Transfer

A REST API uses HTTP methods to perform operations.

REST API Methods

Method	Purpose	Example
GET	Read data	Get users
POST	Create data	Add user
PUT	Update data	Update user
DELETE	Remove data	Delete user

API Data Format

Most APIs use:

JSON

Example:

{
    "name": "John",
    "age": 20
}

Python dictionary equivalent:

{
"name":"John",
"age":20
}

What is FastAPI?

FastAPI is a modern Python framework for creating APIs.

Advantages:

Very fast
Automatic documentation
Type checking
Easy validation
Used in production systems

Installing FastAPI

Install:

pip install fastapi uvicorn

First FastAPI Application

Create:

main.py

Code:

from fastapi import FastAPI


app = FastAPI()


@app.get("/")
def home():

    return {
        "message":"Hello FastAPI"
    }

Running FastAPI

Use:

uvicorn main:app --reload

Output:

Running on http://127.0.0.1:8000

Open:

http://127.0.0.1:8000

Response:

{
"message":"Hello FastAPI"
}

Automatic Documentation

FastAPI automatically creates documentation.

Swagger UI:

/docs

Example:

http://127.0.0.1:8000/docs

Alternative:

/redoc

Creating Routes

GET Request

@app.get("/users")
def users():

    return {
        "users":[
            "John",
            "Alice"
        ]
    }

Path Parameters

Dynamic URLs.

Example:

@app.get("/users/{id}")
def user(id:int):

    return {
        "user_id":id
    }

Visit:

/users/5

Response:

{
"user_id":5
}

Query Parameters

Used for optional values.

Example:

@app.get("/products")
def products(limit:int=10):

    return {
        "limit":limit
    }

URL:

/products?limit=5

Response:

{
"limit":5
}

Request Body

When sending data to server.

Example:

from pydantic import BaseModel


class User(BaseModel):

    name:str
    age:int

POST API

@app.post("/users")
def create_user(user:User):

    return user

Request:

{
"name":"John",
"age":20
}

Response:

{
"name":"John",
"age":20
}

Pydantic Models

Pydantic validates data automatically.

Example:

class Student(BaseModel):

    name:str

    age:int

    marks:float

Correct:

{
"name":"Alex",
"age":20,
"marks":85.5
}

Wrong:

{
"name":"Alex",
"age":"abc"
}

FastAPI returns an error.

PUT Request

Used to update data.

Example:

@app.put("/users/{id}")
def update_user(id:int,user:User):

    return {
        "id":id,
        "data":user
    }

DELETE Request

Example:

@app.delete("/users/{id}")
def delete_user(id:int):

    return {
        "message":"User deleted"
    }

Status Codes

HTTP responses use status codes.

Common codes:

Code	Meaning
200	Success
201	Created
400	Bad Request
401	Unauthorized
404	Not Found
500	Server Error

Returning Custom Status Codes

Example:

from fastapi import status


@app.post("/users",
status_code=status.HTTP_201_CREATED)
def create_user():

    return {
        "message":"Created"
    }

Error Handling

Use:

HTTPException

Example:

from fastapi import HTTPException


@app.get("/users/{id}")
def user(id:int):

    if id!=1:

        raise HTTPException(
            status_code=404,
            detail="User not found"
        )

    return {
        "id":id
    }

Connecting FastAPI with Database

Architecture:

Client

 |

FastAPI

 |

SQL Database

 |

Data

Common databases:

SQLite
MySQL
PostgreSQL

Using SQLAlchemy

Install:

pip install sqlalchemy

SQLAlchemy converts Python classes into database tables.

Example:

from sqlalchemy import Column,Integer,String


class User:

    id = Column(Integer)

    name = Column(String)

API Authentication

Authentication verifies users.

Common methods:

1. API Keys

Example:

Authorization:
API-Key 123456

2. JWT Tokens

JSON Web Token:

User Login

     |

Server creates token

     |

User sends token

     |

Access granted

JWT Example Flow

User enters username/password
Server checks database
Server creates token
Client stores token
Token sent with requests

Middleware

Middleware runs before or after requests.

Used for:

Logging
Authentication
Security
Performance monitoring

Example:

@app.middleware("http")
async def log(request,call_next):

    print("Request received")

    response = await call_next(request)

    return response

Background Tasks

Run tasks after response.

Examples:

Sending emails
Processing files
Notifications

Example:

from fastapi import BackgroundTasks


def send_email():

    print("Email sent")



@app.post("/send")
def send(
background_tasks:BackgroundTasks
):

    background_tasks.add_task(send_email)

    return "Done"

File Upload API

Example:

from fastapi import UploadFile


@app.post("/upload")
def upload(file:UploadFile):

    return {
        "filename":file.filename
    }

Real-World FastAPI Projects

1. E-commerce API

Features:

Products
Users
Orders
Payments

2. Social Media API

Features:

Profiles
Posts
Comments
Likes

3. AI Application Backend

Features:

User requests
Model processing
Results API

Practice Exercises

Create a FastAPI hello world API.
Create user CRUD API.
Create product API.
Connect API with SQLite.
Add authentication.
Create file upload API.

Chapter 26: Python Testing and Debugging

What is Software Testing?

Testing is the process of checking whether a program works correctly.

Testing helps to:

Find bugs
Improve quality
Prevent failures
Make code reliable

Example:

A calculator program should be tested:

add(2,3)

Expected:

Types of Testing

1. Manual Testing

A person checks the application manually.

Example:

Open website
Click buttons
Check results

2. Automated Testing

Programs test other programs.

Benefits:

Faster
Repeatable
Less human effort

Python supports:

unittest
pytest

What is a Bug?

A bug is an error in a program.

Example:

def add(a,b):

    return a-b

Expected:

For:

add(2,3)

Output:

-1

This is a bug.

Unit Testing

What is Unit Testing?

Testing small parts of a program individually.

Example:

Testing only:

One function
One class
One module

Python unittest Module

Python includes:

unittest

No installation required.

Creating a Function to Test

File:

calculator.py

Code:

def add(a,b):

    return a+b


def subtract(a,b):

    return a-b

Creating Test File

File:

test_calculator.py

Code:

import unittest

from calculator import add


class TestCalculator(unittest.TestCase):


    def test_add(self):

        result = add(2,3)

        self.assertEqual(result,5)



if __name__=="__main__":

    unittest.main()

Run:

python test_calculator.py

Output:

OK

Common unittest Methods

Method	Purpose
assertEqual()	Check equality
assertNotEqual()	Check difference
assertTrue()	Check true
assertFalse()	Check false
assertRaises()	Check errors

Example: assertEqual()

self.assertEqual(
add(2,3),
5
)

Testing Exceptions

Example:

def divide(a,b):

    if b==0:

        raise ValueError()

    return a/b

Test:

def test_divide_error(self):

    with self.assertRaises(ValueError):

        divide(10,0)

Test Setup and Cleanup

Sometimes we need preparation before tests.

Use:

setUp()

Example:

class TestDatabase(unittest.TestCase):


    def setUp(self):

        self.name="John"


    def test_name(self):

        self.assertEqual(
            self.name,
            "John"
        )

pytest Framework

pytest is a popular testing framework.

Install:

pip install pytest

Simple pytest Example

File:

test_math.py

Code:

def add(a,b):

    return a+b



def test_add():

    assert add(2,3)==5

Run:

pytest

Output:

1 passed

pytest Advantages

Compared to unittest:

Less code
Simple syntax
Powerful plugins
Better reports

Testing Different Cases

Example:

def multiply(a,b):

    return a*b



def test_multiply():

    assert multiply(2,3)==6

    assert multiply(5,5)==25

Parametrized Testing

Test many values.

Example:

import pytest


@pytest.mark.parametrize(
"input,output",
[
(2,4),
(3,9),
(4,16)
]
)


def test_square(input,output):

    assert input*input==output

Mocking

Sometimes we test code without using real services.

Example:

Instead of:

Real Payment System

Use:

Fake Payment System

This is called mocking.

Debugging

What is Debugging?

Debugging means finding and fixing errors in code.

Types of Bugs

Syntax Bugs

Example:

print("Hello"

Runtime Bugs

Example:

10/0

Logic Bugs

Example:

Wrong calculation.

Using print() for Debugging

Example:

def add(a,b):

    print(a,b)

    return a+b

Python Debugger (pdb)

Python provides:

pdb

Example:

import pdb


x=10

pdb.set_trace()

y=20

print(x+y)

Program pauses and allows inspection.

Debugger Commands

Inside pdb:

Command	Meaning
n	Next line
s	Step inside function
c	Continue
p	Print value
q	Quit

Using breakpoint()

Modern Python:

x=10

breakpoint()

print(x)

Python automatically starts debugger.

Logging

Logging records program activity.

Better than using many print statements.

Python Logging Module

Built-in:

import logging

Example:

import logging


logging.basicConfig(
level=logging.INFO
)


logging.info(
"Program started"
)

Output:

INFO:root:Program started

Logging Levels

Level	Usage
DEBUG	Detailed information
INFO	Normal activity
WARNING	Possible issue
ERROR	Error occurred
CRITICAL	Serious failure

Writing Logs to File

Example:

import logging


logging.basicConfig(
filename="app.log",
level=logging.INFO
)


logging.info(
"Application started"
)

Creates:

app.log

Error Tracking Example

import logging


try:

    result=10/0


except Exception as e:

    logging.error(e)

Code Quality Tools

Professional Python developers use:

1. pylint

Checks code quality.

Install:

pip install pylint

2. black

Automatically formats code.

Install:

pip install black

3. flake8

Checks style errors.

Install:

pip install flake8

Test Driven Development (TDD)

TDD means:

Write test first
Write code
Improve code

Flow:

Write Test
    |
    v
Test Fails
    |
    v
Write Code
    |
    v
Test Passes

Real Project Testing Structure

Example:

project/

│
├── app/

│    └── main.py

│
├── tests/

│    └── test_main.py

│
└── requirements.txt

Practice Exercises

Write tests for a calculator.
Test a login function.
Test file handling functions.
Create pytest test cases.
Debug a program using pdb.
Add logging to an application.

Chapter 27: Python Project Structure and Package Management

What are Modules?

A module is a Python file that contains:

Variables
Functions
Classes
Executable code

Any .py file can be a module.

Example:

File:

math_tools.py

Code:

def add(a,b):

    return a+b


def multiply(a,b):

    return a*b

Importing a Module

Another file:

import math_tools


result = math_tools.add(5,3)

print(result)

Output:

Import Specific Functions

Instead of importing everything:

from math_tools import add


print(add(10,20))

Output:

Import with Alias

Use:

import math_tools as mt


print(mt.add(2,3))

Built-in Python Modules

Python provides many modules.

Examples:

math

import math

print(math.sqrt(25))

Output:

5.0

random

import random

print(random.randint(1,10))

datetime

import datetime


today=datetime.datetime.now()

print(today)

What are Packages?

A package is a collection of modules stored in a folder.

Example:

my_package/

    __init__.py

    math.py

    string.py

Creating a Package

Structure:

project/

│
├── main.py
│
└── tools/

      ├── __init__.py

      └── calculator.py

File:

calculator.py

def add(a,b):

    return a+b

Using Package:

from tools.calculator import add


print(add(5,6))

Output:

What is `init.py`?

It tells Python that a folder is a package.

Example:

tools/

    __init__.py

It can also contain package initialization code.

Python Package Index (PyPI)

PyPI is the official Python package repository.

Website:

Python Package Index (PyPI)

Developers publish packages here.

Examples:

Flask
Django
NumPy
Requests

What is pip?

pip is Python’s package installer.

It installs external libraries.

Installing Packages

Example:

pip install requests

Using Installed Package

import requests


response = requests.get(
"https://example.com"
)


print(response.status_code)

Checking Installed Packages

Command:

pip list

Example output:

Flask
requests
numpy

Removing Packages

Command:

pip uninstall package_name

Example:

pip uninstall requests

Updating Packages

Command:

pip install --upgrade package_name

Example:

pip install --upgrade flask

Virtual Environments

What is a Virtual Environment?

A virtual environment creates an isolated Python environment for a project.

Why?

Different projects may need different package versions.

Example:

Project A:

Django 4

Project B:

Django 5

Virtual environments keep them separate.

Creating Virtual Environment

Command:

python -m venv env

Creates:

env/

Activating Virtual Environment

Windows

env\Scripts\activate

Linux/Mac

source env/bin/activate

After activation:

(env)

appears in terminal.

Deactivate Environment

Command:

deactivate

requirements.txt

A file that stores project dependencies.

Example:

flask==3.0.0
requests==2.31.0
numpy==1.26.0

Creating requirements.txt

Command:

pip freeze > requirements.txt

Installing Dependencies

For another computer:

pip install -r requirements.txt

Professional Project Structure

Example:

my_project/

│
├── app/

│   ├── __init__.py

│   ├── main.py

│   ├── models.py

│   └── database.py

│
├── tests/

│   └── test_app.py

│
├── requirements.txt

├── README.md

└── .gitignore

`name == "main"`

Important Python pattern.

Example:

def main():

    print("Running program")


if __name__=="__main__":

    main()

Why use it?

It allows a file to be:

Run directly
Imported as a module

Creating a Python Library

Example:

Package:

calculator/
│
├── calculator/
│      ├── __init__.py
│      └── operations.py
│
└── setup.py

setup.py

Contains package information.

Example:

from setuptools import setup


setup(

name="calculator",

version="1.0",

packages=["calculator"]

)

Installing Your Own Package

Inside project:

pip install .

Publishing Package to PyPI

Steps:

Create package
Build package
Upload to PyPI

Tools:

pip install build twine

Build:

python -m build

Upload:

twine upload dist/*

Environment Variables

Used for storing sensitive information.

Examples:

Passwords
API keys
Database URLs

Bad:

password="12345"

Good:

import os

password=os.getenv("PASSWORD")

Using `.env` Files

Install:

pip install python-dotenv

File:

.env

Content:

PASSWORD=mysecret

Python:

from dotenv import load_dotenv
import os


load_dotenv()


password=os.getenv("PASSWORD")

Project Documentation

Good projects include:

README.md

Contains:

Project description
Installation steps
Usage instructions

Example:

# My Application

Install:

pip install -r requirements.txt

Run:

python main.py

Version Control with Git

Git tracks code changes.

Common commands:

git init

git add .

git commit -m "first commit"

Practice Exercises

Create your own Python module.
Create a package with multiple files.
Create a virtual environment.
Generate requirements.txt.
Build a small reusable library.
Organize a professional Python project.

Chapter 28: Python Data Science Introduction

What is Data Science?

Data Science is the process of collecting, cleaning, analyzing, and understanding data to make decisions.

Python is one of the most popular languages for data science because of its powerful libraries.

Data Science Workflow

A typical data science process:

Collect Data
      |
      v
Clean Data
      |
      v
Analyze Data
      |
      v
Visualize Data
      |
      v
Build Models
      |
      v
Make Decisions

Important Python Data Science Libraries

Library	Purpose
NumPy	Numerical calculations
Pandas	Data handling
Matplotlib	Data visualization
Seaborn	Statistical graphs
Scikit-learn	Machine learning
TensorFlow	Deep learning

Installing Data Science Libraries

Install:

pip install numpy pandas matplotlib seaborn scikit-learn

1. NumPy Introduction

What is NumPy?

NumPy means:

Numerical Python

It is used for:

Arrays
Mathematical operations
Scientific calculations

Import:

import numpy as np

Creating NumPy Arrays

Python List

numbers = [1,2,3,4]

NumPy Array

import numpy as np


array = np.array([1,2,3,4])


print(array)

Output:

[1 2 3 4]

Why Use NumPy?

Normal Python:

a=[1,2,3]
b=[4,5,6]

NumPy:

a=np.array([1,2,3])
b=np.array([4,5,6])

print(a+b)

Output:

[5 7 9]

NumPy performs calculations faster.

Array Dimensions

One Dimensional Array

a=np.array([1,2,3])

Shape:

(3,)

Two Dimensional Array

Matrix:

a=np.array(
[
[1,2],
[3,4]
]
)

print(a)

Output:

[[1 2]
 [3 4]]

Checking Array Properties

Example:

print(a.shape)

print(a.size)

print(a.dtype)

Creating Special Arrays

Zeros

np.zeros(5)

Output:

[0. 0. 0. 0. 0.]

Ones

np.ones(5)

Output:

[1. 1. 1. 1. 1.]

Range

np.arange(1,10)

Output:

[1 2 3 4 5 6 7 8 9]

Array Operations

Example:

a=np.array([1,2,3])


print(a*2)

Output:

[2 4 6]

Mathematical Functions

Sum

a=np.array([1,2,3])

print(np.sum(a))

Output:

Mean

print(np.mean(a))

Output:

Maximum

print(np.max(a))

Output:

2. Pandas Introduction

What is Pandas?

Pandas is used for:

Reading data
Cleaning data
Analyzing data
Working with tables

Import:

import pandas as pd

Pandas Series

A Series is a one-dimensional data structure.

Example:

import pandas as pd


data=pd.Series(
[10,20,30]
)


print(data)

Output:

0    10
1    20
2    30

Pandas DataFrame

A DataFrame is like a table.

Example:

import pandas as pd


data={

"Name":["John","Alice","Bob"],

"Age":[20,22,21]

}


df=pd.DataFrame(data)


print(df)

Output:

    Name   Age
0   John   20
1   Alice  22
2   Bob    21

Accessing Columns

Example:

print(df["Name"])

Output:

John
Alice
Bob

Accessing Rows

Using loc:

print(df.loc[0])

Output:

Name John
Age 20

Reading CSV Files

CSV files store tabular data.

Example:

students.csv

Read:

df=pd.read_csv(
"students.csv"
)

print(df)

Writing CSV Files

df.to_csv(
"output.csv"
)

Viewing Data

First rows:

df.head()

Last rows:

df.tail()

Information:

df.info()

Statistics:

df.describe()

Data Cleaning

Real-world data often contains:

Missing values
Duplicate data
Wrong formats

Finding Missing Values

Example:

df.isnull()

Removing Missing Values

df.dropna()

Filling Missing Values

df.fillna(0)

Removing Duplicates

df.drop_duplicates()

Filtering Data

Example:

students = df[
df["Age"] > 20
]

print(students)

Sorting Data

Example:

df.sort_values(
"Age"
)

Adding New Column

Example:

df["Passed"] = True

Removing Column

df.drop(
"Passed",
axis=1
)

Grouping Data

Example:

df.groupby(
"Department"
).mean()

Used for:

Reports
Statistics
Business analysis

3. Matplotlib Introduction

What is Matplotlib?

Matplotlib creates charts and graphs.

Import:

import matplotlib.pyplot as plt

Line Chart

Example:

import matplotlib.pyplot as plt


x=[1,2,3,4]

y=[10,20,30,40]


plt.plot(x,y)


plt.show()

Bar Chart

names=[
"John",
"Alice",
"Bob"
]

marks=[
80,
90,
75
]


plt.bar(
names,
marks
)


plt.show()

Scatter Plot

Used to show relationships.

plt.scatter(
x,
y
)

plt.show()

Histogram

Shows distribution.

plt.hist(
data
)

plt.show()

Adding Labels

Example:

plt.xlabel("Time")

plt.ylabel("Value")

plt.title("Chart")

Real Data Science Projects

1. Sales Analysis

Tasks:

Load sales data
Clean data
Find trends
Create graphs

2. Student Performance Analysis

Tasks:

Analyze marks
Find averages
Create reports

3. Customer Analysis

Tasks:

Customer behavior
Spending patterns
Business insights

Practice Exercises

Create NumPy arrays.
Perform mathematical operations.
Create a Pandas DataFrame.
Read a CSV file.
Clean missing data.
Create line and bar charts.
Analyze a small dataset.

Chapter 29: Python Machine Learning Basics

What is Machine Learning?

Machine Learning (ML) is a branch of Artificial Intelligence (AI) where computers learn patterns from data and make predictions or decisions without being explicitly programmed.

Example:

Traditional Programming:

Rules + Data → Output

Machine Learning:

Data + Output → Learn Rules

AI vs ML vs Deep Learning

Artificial Intelligence (AI)
        |
        |
Machine Learning (ML)
        |
        |
Deep Learning (DL)

Artificial Intelligence

Creating machines that behave intelligently.

Examples:

Chatbots
Self-driving cars
Voice assistants

Machine Learning

Learning from data.

Examples:

Spam detection
Price prediction
Recommendation systems

Deep Learning

Uses neural networks.

Examples:

Image recognition
Speech recognition
Generative AI

Types of Machine Learning

There are three main types:

Supervised Learning
Unsupervised Learning
Reinforcement Learning

1. Supervised Learning

The model learns from labeled data.

Example:

Training data:

House Size	Price
1000 sq ft	50000
2000 sq ft	90000

The model learns:

Size → Price

Types:

Regression

Predicts numbers.

Examples:

House price
Salary prediction
Temperature prediction

Classification

Predicts categories.

Examples:

Spam / Not Spam
Disease / Healthy
Cat / Dog

2. Unsupervised Learning

The model finds patterns without labels.

Example:

Customer grouping:

Customers
     |
     |
Groups discovered automatically

Uses:

Customer segmentation
Pattern discovery

3. Reinforcement Learning

Learning through rewards and penalties.

Example:

A robot learns:

Action → Reward

Used in:

Games
Robotics
Autonomous systems

Machine Learning Workflow

Collect Data

      |

Clean Data

      |

Prepare Features

      |

Train Model

      |

Test Model

      |

Evaluate

      |

Deploy

Installing Scikit-Learn

Scikit-learn is the main ML library for Python.

Install:

pip install scikit-learn

Import:

import sklearn

Machine Learning Terminology

Dataset

Collection of data.

Example:

students.csv

Features (X)

Input values.

Example:

Hours studied
Attendance
Practice tests

Target (y)

Output we want to predict.

Example:

Exam score

Example Dataset

Hours	Score
2	50
4	70
6	90

Features:

X = Hours

Target:

y = Score

Loading Data

Example:

import pandas as pd


data = pd.read_csv(
"students.csv"
)


print(data)

Separating Features and Target

Example:

X = data[
["hours"]
]


y = data[
"score"
]

Training and Testing Data

We split data into:

Training Data

Used for learning.

Testing Data

Used for checking performance.

Usually:

80% Training

20% Testing

train_test_split()

Example:

from sklearn.model_selection import train_test_split


X_train, X_test, y_train, y_test = train_test_split(

X,

y,

test_size=0.2

)

First Machine Learning Model

Linear Regression

Used for predicting numbers.

Example:

Predict salary based on experience.

Import Model

from sklearn.linear_model import LinearRegression

Create Model

model = LinearRegression()

Train Model

model.fit(
X_train,
y_train
)

Make Prediction

prediction = model.predict(
X_test
)


print(prediction)

Complete Example

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.linear_model import LinearRegression



data = pd.DataFrame({

"hours":[1,2,3,4,5],

"score":[20,40,60,80,100]

})


X=data[["hours"]]

y=data["score"]



X_train,X_test,y_train,y_test = train_test_split(
X,
y,
test_size=0.2
)



model=LinearRegression()


model.fit(
X_train,
y_train
)



result=model.predict(
[[6]]
)


print(result)

Output:

Classification Example

Logistic Regression

Used for categories.

Example:

Email
 |
 |
Spam or Not Spam

Import:

from sklearn.linear_model import LogisticRegression

Create:

model = LogisticRegression()

Train:

model.fit(
X_train,
y_train
)

Predict:

model.predict(
X_test
)

Popular Machine Learning Algorithms

Regression

Linear Regression

Predicts continuous values.

Example:

Price prediction

Polynomial Regression

Handles curved relationships.

Classification

Logistic Regression

Binary classification.

Example:

Yes / No

Decision Tree

Makes decisions using rules.

Example:

Age > 18?
 |
Yes → Adult
No → Child

Random Forest

Collection of decision trees.

Advantages:

Accurate
Handles complex data

Support Vector Machine (SVM)

Finds boundaries between classes.

Clustering

K-Means

Groups similar data.

Example:

Customers

Group A
Group B
Group C

Model Evaluation

A model must be tested.

Regression Metrics

Mean Absolute Error (MAE)

Measures average error.

from sklearn.metrics import mean_absolute_error


error = mean_absolute_error(
y_test,
prediction
)

R² Score

Measures model accuracy.

from sklearn.metrics import r2_score


score = r2_score(
y_test,
prediction
)

Classification Metrics

Accuracy

Percentage of correct predictions.

from sklearn.metrics import accuracy_score

Confusion Matrix

Shows:

Correct predictions
Wrong predictions

from sklearn.metrics import confusion_matrix

Feature Scaling

Some algorithms require similar value ranges.

Example:

Before:

Age: 20
Salary: 500000

After scaling:

Age: 0.2
Salary: 0.5

StandardScaler

Example:

from sklearn.preprocessing import StandardScaler


scaler = StandardScaler()


X_scaled = scaler.fit_transform(X)

Saving Machine Learning Models

Use:

joblib

Install:

pip install joblib

Save:

import joblib


joblib.dump(
model,
"model.pkl"
)

Load:

model = joblib.load(
"model.pkl"
)

Real Machine Learning Projects

1. House Price Prediction

Features:

Area
Rooms
Location

Output:

Price

2. Spam Detection

Features:

Email text

Output:

Spam / Not spam

3. Customer Segmentation

Features:

Age
Income
Purchase history

Output:

Customer groups

Practice Exercises

Create a linear regression model.
Predict house prices.
Build a spam classifier.
Train a decision tree model.
Calculate model accuracy.
Save and load a trained model.

Chapter 30: Python Deep Learning and Neural Networks

What is Deep Learning?

Deep Learning is a branch of Machine Learning that uses artificial neural networks to learn complex patterns from large amounts of data.

Deep Learning is inspired by the human brain.

Examples:

Image recognition
Voice assistants
Self-driving cars
Language translation
Generative AI

AI → ML → Deep Learning

Artificial Intelligence
          |
          |
   Machine Learning
          |
          |
    Deep Learning

Machine Learning vs Deep Learning

Machine Learning	Deep Learning
Needs feature selection	Learns features automatically
Works with smaller data	Needs large data
Faster training	More computation
Uses algorithms	Uses neural networks

What is an Artificial Neural Network?

A neural network is a system made of connected artificial neurons.

Structure:

Input Layer
     |
     |
Hidden Layers
     |
     |
Output Layer

Artificial Neuron

A neuron receives inputs, applies weights, and produces output.

Formula:

Output = Activation(Input × Weight + Bias)

Neural Network Components

1. Input Layer

Receives data.

Example:

Image pixels:

Pixel 1
Pixel 2
Pixel 3

2. Hidden Layers

Learn patterns.

Examples:

First layer:

Edges

Second layer:

Shapes

Third layer:

Objects

3. Output Layer

Produces final result.

Example:

Cat = 90%
Dog = 10%

Installing TensorFlow

TensorFlow is a popular deep learning library.

Install:

pip install tensorflow

Import:

import tensorflow as tf

Keras

Keras is a high-level API inside TensorFlow.

It makes neural networks easier to build.

Import:

from tensorflow import keras

Creating Your First Neural Network

Example:

import tensorflow as tf


model = tf.keras.Sequential([
    
    tf.keras.layers.Dense(10),
    
    tf.keras.layers.Dense(1)

])


print(model)

Understanding Dense Layer

Dense means:

Every neuron connects to every neuron in the next layer.

Example:

Neuron A ----\
Neuron B ----- Output
Neuron C ----/

Activation Functions

Activation functions decide whether a neuron activates.

Common functions:

ReLU
Sigmoid
Softmax
Tanh

ReLU Function

Most common in hidden layers.

Formula:

if x > 0:
    x
else:
    0

Example:

tf.keras.layers.Dense(
10,
activation="relu"
)

Sigmoid Function

Used for binary classification.

Output:

0 to 1

Example:

Spam probability = 0.95

Softmax Function

Used for multiple categories.

Example:

Image classification:

Cat: 0.7
Dog: 0.2
Bird:0.1

Creating a Complete Model

Example:

import tensorflow as tf


model = tf.keras.Sequential([

    tf.keras.layers.Dense(
        64,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        10,
        activation="softmax"
    )

])

Compiling a Model

Before training:

model.compile(

optimizer="adam",

loss="categorical_crossentropy",

metrics=["accuracy"]

)

Important Training Terms

Epoch

One complete pass through training data.

Example:

1000 images

1 epoch = model sees all 1000 images once

Batch Size

Number of samples processed at once.

Example:

100 images

batch size = 10

10 batches

Loss Function

Measures how wrong the model is.

Goal:

Reduce loss

Optimizer

Updates model weights.

Popular:

Adam
SGD
RMSProp

Training a Neural Network

Example:

model.fit(

X_train,

y_train,

epochs=10

)

The model learns patterns.

Prediction

After training:

prediction = model.predict(
X_test
)

Example: Handwritten Digit Recognition

Dataset:

MNIST

Contains:

70,000 handwritten digits

Classes:

0 1 2 3 4 5 6 7 8 9

Loading MNIST Dataset

from tensorflow.keras.datasets import mnist


(X_train,y_train),(X_test,y_test)=mnist.load_data()

Understanding Image Data

Image:

28 x 28 pixels

Shape:

print(X_train.shape)

Output:

(60000,28,28)

Normalizing Data

Convert:

0-255

into:

0-1

Example:

X_train = X_train / 255.0

X_test = X_test / 255.0

Building Digit Classifier

model=tf.keras.Sequential([


tf.keras.layers.Flatten(
input_shape=(28,28)
),


tf.keras.layers.Dense(
128,
activation="relu"
),


tf.keras.layers.Dense(
10,
activation="softmax"
)


])

Training

model.compile(

optimizer="adam",

loss="sparse_categorical_crossentropy",

metrics=["accuracy"]

)


model.fit(

X_train,

y_train,

epochs=5

)

Evaluating Model

model.evaluate(
X_test,
y_test
)

Output:

Accuracy: 97%

Convolutional Neural Networks (CNN)

CNNs are used mainly for images.

Applications:

Face recognition
Medical images
Object detection

CNN Structure

Image

 |

Convolution Layer

 |

Pooling Layer

 |

Dense Layer

 |

Output

Convolution Layer

Finds:

Edges
Shapes
Patterns

Example:

Image → Features

Pooling Layer

Reduces size while keeping important information.

Benefits:

Faster processing
Less memory

Recurrent Neural Networks (RNN)

Used for sequential data.

Examples:

Text
Speech
Time series

Long Short-Term Memory (LSTM)

A type of RNN.

Used for:

Language models
Translation
Predictions

Natural Language Processing (NLP)

NLP allows computers to understand human language.

Applications:

Chatbots
Translation
Sentiment analysis

Text Processing Steps

Text

 |

Tokenization

 |

Convert words to numbers

 |

Train Model

 |

Prediction

Tokenization Example

Sentence:

I love Python

Converted:

[1,2,3]

Word Embeddings

Convert words into numerical vectors.

Example:

Python → [0.23,0.76,0.11]

Words with similar meaning have similar vectors.

Transfer Learning

Using an already trained model.

Example:

Instead of training image recognition from zero:

Use:

ResNet
MobileNet
VGG

Benefits:

Faster training
Better results

Deep Learning Projects

1. Image Classifier

Input:

Image

Output:

Category

2. Face Recognition System

Uses:

CNN
Computer Vision

3. Chatbot

Uses:

NLP
Neural Networks

4. Object Detection

Detects:

Cars
People
Objects

Practice Exercises

Install TensorFlow.
Create a simple neural network.
Train a digit classifier.
Experiment with activation functions.
Build an image classifier.
Create a text classification model.

Chapter 31: Python Computer Vision with OpenCV

What is Computer Vision?

Computer Vision is a field of Artificial Intelligence that allows computers to understand and process images and videos.

Humans use eyes to see.

Computers use:

Cameras
Images
Algorithms
Neural networks

to understand the world.

Applications of Computer Vision

Examples:

Face recognition
Object detection
Medical image analysis
Self-driving cars
Security cameras
OCR (text reading)
Augmented Reality

What is OpenCV?

OpenCV (Open Source Computer Vision Library) is a popular library for image and video processing.

Python uses:

cv2

Installing OpenCV

Install:

pip install opencv-python

Import:

import cv2

Understanding Images

A digital image is made of pixels.

Example:

Image

+---+---+---+
|   |   |   |
+---+---+---+
|   |   |   |
+---+---+---+

Each pixel stores color information.

Image Color Formats

RGB

Three channels:

R = Red
G = Green
B = Blue

Example:

(255,0,0)

means red.

Grayscale

Only brightness values:

0 = Black
255 = White

Reading an Image

Example:

import cv2


image = cv2.imread(
"photo.jpg"
)


print(image)

Displaying an Image

import cv2


image = cv2.imread(
"photo.jpg"
)


cv2.imshow(
"Image",
image
)


cv2.waitKey(0)

cv2.destroyAllWindows()

Saving an Image

cv2.imwrite(
"new_photo.jpg",
image
)

Image Properties

Example:

print(image.shape)

Output:

(height, width, channels)

Example:

(500,700,3)

Means:

Height = 500 pixels
Width = 700 pixels
RGB channels = 3

Resizing Images

Example:

small = cv2.resize(

image,

(300,300)

)

Changing Image Size Percentage

Example:

small = cv2.resize(

image,

None,

fx=0.5,

fy=0.5

)

Reduces size by 50%.

Cropping Images

Images are arrays.

Example:

crop = image[
100:300,
100:300
]

Meaning:

height range
width range

Rotating Images

Example:

rotated = cv2.rotate(

image,

cv2.ROTATE_90_CLOCKWISE

)

Converting Color Spaces

RGB to Grayscale

gray = cv2.cvtColor(

image,

cv2.COLOR_BGR2GRAY

)

Image Filtering

Filters improve or modify images.

Used for:

Removing noise
Smoothing
Detecting edges

Gaussian Blur

Example:

blur = cv2.GaussianBlur(

image,

(5,5),

0

)

Edge Detection

Edges identify boundaries.

Example:

edges = cv2.Canny(

image,

100,

200

)

Thresholding

Converts image into black and white.

Example:

_,binary = cv2.threshold(

gray,

127,

255,

cv2.THRESH_BINARY

)

Drawing Shapes

OpenCV can draw:

Lines
Rectangles
Circles
Text

Drawing a Rectangle

cv2.rectangle(

image,

(50,50),

(200,200),

(255,0,0),

2

)

Drawing a Circle

cv2.circle(

image,

(100,100),

50,

(0,255,0),

2

)

Adding Text

cv2.putText(

image,

"Hello",

(50,50),

cv2.FONT_HERSHEY_SIMPLEX,

1,

(255,255,255),

2

)

Video Processing

A video is a sequence of images called frames.

Structure:

Frame 1
Frame 2
Frame 3
...

Reading Webcam

Example:

import cv2


camera = cv2.VideoCapture(0)


while True:

    ret,frame = camera.read()


    cv2.imshow(
    "Camera",
    frame
    )


    if cv2.waitKey(1)==ord("q"):

        break


camera.release()

cv2.destroyAllWindows()

Face Detection

OpenCV provides pretrained models.

Common method:

Haar Cascade

Used for:

Face detection
Eye detection

Loading Face Detector

face = cv2.CascadeClassifier(

"haarcascade_frontalface_default.xml"

)

Detecting Faces

faces = face.detectMultiScale(

gray,

1.3,

5

)

Returns face locations.

Drawing Face Boxes

for x,y,w,h in faces:

    cv2.rectangle(

    image,

    (x,y),

    (x+w,y+h),

    (255,0,0),

    2

    )

Object Detection

Object detection identifies objects in images.

Examples:

Cars
People
Animals

Popular models:

YOLO
SSD
Faster R-CNN

YOLO (You Only Look Once)

YOLO is a popular real-time object detection algorithm.

It can detect:

Image

Person 90%

Car 85%

Dog 92%

OCR (Reading Text from Images)

OCR means:

Optical Character Recognition

Used for:

Scanning documents
Reading number plates
Extracting text

Python library:

pip install pytesseract

OCR Example

import pytesseract


text = pytesseract.image_to_string(
image
)


print(text)

Image Classification

Using Deep Learning:

Image

 |

CNN Model

 |

Prediction

 |

Cat

Libraries:

TensorFlow
PyTorch
Keras

Real Computer Vision Projects

1. Face Recognition System

Features:

Detect faces
Compare faces
Identify people

2. Attendance System

Features:

Camera input
Face detection
Database storage

3. Number Plate Recognition

Features:

Detect vehicle
Read plate text
Store information

4. Object Detection Camera

Features:

Live video
Detect objects
Display labels

Practice Exercises

Read and display an image.
Resize an image.
Convert image to grayscale.
Detect edges using Canny.
Capture webcam video.
Detect faces.
Extract text from an image.

Chapter 32: Python Automation and Scripting

What is Automation?

Automation means making a computer perform tasks automatically without manual effort.

Python is widely used for automation because it can:

Control files
Work with websites
Send emails
Manage spreadsheets
Process data
Schedule tasks

Why Use Python Automation?

Without automation:

Open file
Read data
Copy information
Create report
Send email

Manually every day.

With Python:

Run Script
    |
    |
Complete all tasks automatically

Common Automation Areas

1. File Automation

Examples:

Rename files
Move files
Delete old files
Create folders

2. Web Automation

Examples:

Open websites
Fill forms
Download reports

Tools:

Selenium
Playwright

3. Data Automation

Examples:

Excel reports
CSV processing
Data cleaning

Tools:

Pandas
OpenPyXL

4. Communication Automation

Examples:

Send emails
Send notifications
Generate messages

Working with Files

Python provides:

os

module for operating system tasks.

Import:

import os

Checking Current Folder

import os


location = os.getcwd()

print(location)

Output:

/home/user/project

Creating a Folder

import os


os.mkdir("new_folder")

Creates:

new_folder/

Listing Files

import os


files = os.listdir(".")


print(files)

Example output:

['app.py','data.csv']

Checking File Exists

import os


if os.path.exists("data.txt"):

    print("File exists")

else:

    print("Not found")

Renaming Files

import os


os.rename(
"old.txt",
"new.txt"
)

Moving Files

Use:

shutil

Example:

import shutil


shutil.move(
"file.txt",
"folder/"
)

Copying Files

import shutil


shutil.copy(
"a.txt",
"backup/"
)

Deleting Files

import os


os.remove(
"file.txt"
)

Reading Multiple Files Automatically

Example:

import os


folder="reports"


for file in os.listdir(folder):

    print(file)

Automating CSV Files

Install:

pip install pandas

Example:

import pandas as pd


data=pd.read_csv(
"sales.csv"
)


print(data.head())

Creating Excel Files

Install:

pip install openpyxl

Example:

from openpyxl import Workbook


book = Workbook()


sheet = book.active


sheet["A1"]="Name"

sheet["B1"]="Age"


book.save(
"students.xlsx"
)

Reading Excel Files

from openpyxl import load_workbook


book = load_workbook(
"students.xlsx"
)


sheet=book.active


print(sheet["A1"].value)

Automating Reports

Example workflow:

Read Data
    |
Calculate Results
    |
Create Excel Report
    |
Send Email

Sending Emails with Python

Python provides:

smtplib

module.

Example:

import smtplib


server=smtplib.SMTP(
"smtp.gmail.com",
587
)


server.starttls()


server.login(
"email",
"password"
)


server.sendmail(
"from",
"to",
"Hello"
)


server.quit()

Email with Attachments

Libraries:

email
smtplib

Used for:

Reports
Invoices
Notifications

Web Scraping

What is Web Scraping?

Automatically collecting information from websites.

Examples:

Price tracking
News collection
Data gathering

BeautifulSoup

Install:

pip install beautifulsoup4 requests

Import:

from bs4 import BeautifulSoup
import requests

Getting Web Page Data

Example:

import requests


url="https://example.com"


response=requests.get(url)


print(response.text)

Parsing HTML

from bs4 import BeautifulSoup


soup=BeautifulSoup(
response.text,
"html.parser"
)


print(soup.title)

Finding Elements

Example:

links=soup.find_all("a")


for link in links:

    print(link.text)

Browser Automation with Selenium

What is Selenium?

Selenium controls browsers automatically.

Used for:

Testing websites
Filling forms
Clicking buttons

Installing Selenium

pip install selenium

Opening Browser

Example:

from selenium import webdriver


browser = webdriver.Chrome()


browser.get(
"https://google.com"
)

Finding Elements

Example:

search = browser.find_element(
"id",
"search"
)

Clicking Buttons

button.click()

Entering Text

search.send_keys(
"Python"
)

Closing Browser

browser.quit()

Scheduling Automation

Python can run tasks automatically.

Library:

schedule

Install:

pip install schedule

Scheduled Task Example

import schedule
import time


def job():

    print(
    "Task Running"
    )


schedule.every().day.at(
"10:00"
).do(job)


while True:

    schedule.run_pending()

    time.sleep(1)

Working with APIs

Automation often uses APIs.

Example:

Python Script

     |

API Request

     |

Receive Data

     |

Process Automatically

Creating Automation Scripts

Good automation scripts have:

1. Configuration

Settings:

FILE_PATH="data.csv"

2. Functions

Example:

def process_file():

    pass

3. Error Handling

Example:

try:

    process()

except Exception as e:

    print(e)

Real Automation Projects

1. Automatic File Organizer

Features:

Detect file type
Create folders
Move files

Example:

Downloads

 |

Images
Documents
Videos

2. Email Report Generator

Features:

Read data
Create report
Send email

3. Website Monitoring Bot

Features:

Check website
Detect changes
Send alert

4. Invoice Generator

Features:

Read customer data
Create PDF
Email invoice

Best Practices

Use Virtual Environment

python -m venv env

Add Logging

import logging

logging.info(
"Task completed"
)

Handle Errors

try:

    task()

except Exception:

    print("Failed")

Keep Code Organized

Example:

automation_project/

│

├── main.py

├── config.py

├── utils.py

└── logs/

Practice Exercises

Create a file organizer.
Automate Excel report creation.
Create a web scraper.
Send automatic emails.
Build a scheduled backup script.
Automate browser actions.

Chapter 33: Python Databases and SQL Integration

What is a Database?

A database is a system used to store, organize, and manage data.

Examples:

User accounts
Products
Orders
Student records
Financial information

Why Use Databases?

Without a database:

Python Program
      |
      |
Data stored in files

Problems:

Difficult searching
Slow with large data
Data duplication

With a database:

Python Program
       |
       |
   Database
       |
       |
 Organized Data

Advantages:

Fast searching
Secure storage
Multiple users
Easy updates

Types of Databases

1. Relational Databases (SQL)

Data stored in tables.

Examples:

SQLite
MySQL
PostgreSQL
Oracle

Structure:

Table: Users

+----+-------+-----+
| ID | Name  | Age |
+----+-------+-----+
| 1  | John  | 20  |
| 2  | Alice | 22  |
+----+-------+-----+

2. NoSQL Databases

Store flexible data.

Examples:

MongoDB
Redis
Cassandra

Example:

{
"name":"John",
"age":20
}

SQL Basics

SQL means:

Structured Query Language

Used to communicate with databases.

Common SQL Commands

Command	Purpose
SELECT	Read data
INSERT	Add data
UPDATE	Modify data
DELETE	Remove data
CREATE	Create table

Creating a Database

Example:

CREATE DATABASE company;

Creating a Table

Example:

CREATE TABLE users(

id INTEGER,

name TEXT,

age INTEGER

);

Inserting Data

Example:

INSERT INTO users

VALUES

(1,'John',20);

Reading Data

Use:

SELECT * FROM users;

Output:

1 John 20

Filtering Data

Example:

SELECT *

FROM users

WHERE age > 18;

Updating Data

Example:

UPDATE users

SET age=21

WHERE id=1;

Deleting Data

Example:

DELETE FROM users

WHERE id=1;

SQLite with Python

SQLite is a lightweight database included with Python.

No installation required.

Import:

import sqlite3

Connecting to SQLite

Example:

import sqlite3


connection = sqlite3.connect(
"company.db"
)


print("Connected")

Creates:

company.db

Creating a Cursor

Cursor executes SQL commands.

Example:

cursor = connection.cursor()

Creating a Table

Example:

cursor.execute(
"""
CREATE TABLE users(

id INTEGER,

name TEXT,

age INTEGER

)
"""
)

Inserting Data Using Python

Example:

cursor.execute(

"""
INSERT INTO users

VALUES(1,'John',20)

"""

)


connection.commit()

Reading Data

Example:

cursor.execute(
"SELECT * FROM users"
)


data = cursor.fetchall()


print(data)

Output:

[(1,'John',20)]

Using Variables in SQL

Avoid:

"INSERT INTO users VALUES(1,'John')"

Better:

cursor.execute(

"INSERT INTO users VALUES(?,?)",

(1,"John")

)

CRUD Operations

CRUD means:

C → Create
R → Read
U → Update
D → Delete

Create

def add_user(name,age):

    cursor.execute(

    "INSERT INTO users VALUES(NULL,?,?)",

    (name,age)

    )

    connection.commit()

Read

def get_users():

    cursor.execute(
    "SELECT * FROM users"
    )

    return cursor.fetchall()

Update

def update_user(id,age):

    cursor.execute(

    "UPDATE users SET age=? WHERE id=?",

    (age,id)

    )

    connection.commit()

Delete

def delete_user(id):

    cursor.execute(

    "DELETE FROM users WHERE id=?",

    (id,)

    )

    connection.commit()

MySQL with Python

MySQL is a popular production database.

Install connector:

pip install mysql-connector-python

Connecting MySQL

Example:

import mysql.connector


db=mysql.connector.connect(

host="localhost",

user="root",

password="password",

database="company"

)


print("Connected")

Executing MySQL Query

cursor=db.cursor()


cursor.execute(
"SELECT * FROM users"
)


result=cursor.fetchall()


print(result)

PostgreSQL with Python

Install:

pip install psycopg2

Connect:

import psycopg2


connection=psycopg2.connect(

database="company",

user="postgres",

password="password"

)

What is ORM?

ORM means:

Object Relational Mapping

It allows you to work with databases using Python objects instead of SQL.

Without ORM:

SQL Query
     |
Database

With ORM:

Python Class
       |
       |
Database Table

SQLAlchemy

SQLAlchemy is a popular Python ORM.

Install:

pip install sqlalchemy

Creating Database Model

Example:

from sqlalchemy import Column,Integer,String


class User:

    id = Column(Integer)

    name = Column(String)

    age = Column(Integer)

Benefits of ORM

Advantages:

Less SQL writing
Cleaner code
Database independent
Easier maintenance

SQLAlchemy Example

Create engine:

from sqlalchemy import create_engine


engine=create_engine(
"sqlite:///company.db"
)

Database Relationships

Real applications have related tables.

Example:

Users

  |

Orders

  |

Products

One-to-One Relationship

Example:

Person
 |
Passport

One person has one passport.

One-to-Many Relationship

Example:

Customer

   |

Many Orders

Many-to-Many Relationship

Example:

Students

   |

Courses

A student can join many courses.

Database Security

Important practices:

1. Use Password Protection

Never store passwords directly.

Bad:

password="12345"

2. Use Password Hashing

Libraries:

bcrypt
passlib

3. Prevent SQL Injection

Bad:

query = "SELECT * FROM users WHERE name='"+name+"'"

Good:

cursor.execute(
"SELECT * FROM users WHERE name=?",
(name,)
)

Database Backup

Important for:

Business data
User information
Applications

Methods:

Export database
Automated backups
Cloud storage

Real Database Projects

1. User Management System

Features:

Registration
Login
Profile

Database:

Users Table

2. E-Commerce Database

Tables:

Users

Products

Orders

Payments

3. School Management System

Tables:

Students

Teachers

Classes

Marks

Practice Exercises

Create SQLite database.
Create users table.
Insert records.
Read records.
Update records.
Delete records.
Build CRUD application.
Connect Python with MySQL.

Chapter 34: Python Security and Authentication

What is Application Security?

Application Security means protecting software, data, and users from unauthorized access and attacks.

Security is important for:

Websites
APIs
Mobile applications
Banking systems
Business applications

Common Security Problems

1. Weak Passwords

Bad example:

password123
123456
admin

Problems:

Easy to guess
Can be stolen

2. Storing Passwords Directly

Wrong:

username="john"

password="mypassword"

If the database is leaked, passwords are exposed.

3. SQL Injection

Attackers insert SQL commands into input fields.

Example:

Username:
admin' OR '1'='1

Solution:

Use prepared statements
Use ORM
Validate input

4. Data Exposure

Sensitive information should not be visible.

Examples:

Passwords
API keys
Credit card data

Authentication vs Authorization

Authentication

Answers:

Who are you?

Example:

Username + Password
        |
        |
    Verify User

Authorization

Answers:

What can you access?

Example:

Admin → Delete users

User → View profile

Password Hashing

What is Hashing?

Hashing converts data into a fixed encrypted-looking value.

Example:

Original:

mypassword

Hash:

5f4dcc3b5aa765d61d8327deb882cf99

Hashing Properties

Good hashing should be:

One-way
Secure
Difficult to reverse

Python Password Hashing Libraries

Popular:

bcrypt
passlib

Installing bcrypt

pip install bcrypt

Creating Password Hash

Example:

import bcrypt


password = b"mypassword"


hashed = bcrypt.hashpw(

password,

bcrypt.gensalt()

)


print(hashed)

Output:

b'$2b$12$.......'

Checking Password

Example:

result = bcrypt.checkpw(

b"mypassword",

hashed

)


print(result)

Output:

True

Salting

A salt adds random data before hashing.

Without salt:

password
        |
        |
same hash every time

With salt:

password + random salt
        |
        |
different hashes

Benefits:

Prevents rainbow table attacks
Improves security

User Registration System

Flow:

User enters details

        |

Password hashed

        |

Store hash in database

        |

Account created

Example:

def register(password):

    hashed = bcrypt.hashpw(

    password.encode(),

    bcrypt.gensalt()

    )

    return hashed

Login System

Flow:

User enters password

        |

Compare with stored hash

        |

Access granted

Sessions

A session stores information about a logged-in user.

Example:

Login

 |

Create Session

 |

User accesses pages

Flask Session Example

from flask import session


session["username"]="John"

Access:

print(session["username"])

Cookies

Cookies store small information in browsers.

Examples:

Login status
Preferences
Language

Example:

Browser

 |

Cookie

 |

Server

JSON Web Token (JWT)

JWT is commonly used for APIs.

JWT allows users to prove identity without storing sessions on the server.

JWT Structure

A JWT has three parts:

Header.Payload.Signature

Example:

xxxxx.yyyyy.zzzzz

JWT Authentication Flow

User Login

     |

Server verifies password

     |

Server creates JWT Token

     |

User sends token with requests

     |

Server verifies token

     |

Access granted

Installing JWT Library

For Python:

pip install pyjwt

Creating JWT Token

Example:

import jwt


payload = {

"user":"John"

}


token = jwt.encode(

payload,

"secret_key",

algorithm="HS256"

)


print(token)

Decoding JWT Token

Example:

data = jwt.decode(

token,

"secret_key",

algorithms=["HS256"]

)


print(data)

FastAPI JWT Authentication

Typical structure:

Client

 |

Login API

 |

JWT Token

 |

Protected API

API Key Authentication

Used for:

External APIs
Developer access

Example:

Authorization:
API-Key abc123

Environment Variables

Never store secrets in code.

Bad:

API_KEY="123456"

Good:

import os


key=os.getenv(
"API_KEY"
)

Using .env Files

Install:

pip install python-dotenv

File:

.env

Content:

SECRET_KEY=mysecret
DATABASE_PASSWORD=password

Python:

from dotenv import load_dotenv

import os


load_dotenv()


secret=os.getenv(
"SECRET_KEY"
)

Encryption Basics

Hashing vs Encryption

Hashing	Encryption
One-way	Two-way
Password storage	Data protection
Cannot decrypt	Can decrypt

Symmetric Encryption

Same key used for:

Encryption
Decryption

Example:

Message

 |

Key

 |

Encrypted Data

Library:

cryptography

Installing Cryptography

pip install cryptography

Encryption Example

from cryptography.fernet import Fernet


key = Fernet.generate_key()


cipher = Fernet(key)


encrypted = cipher.encrypt(
b"Secret data"
)


print(encrypted)

Decryption Example

decrypted = cipher.decrypt(
encrypted
)


print(decrypted)

Input Validation

Never trust user input.

Example:

Bad:

age=input()

Good:

age=int(input())

Secure Password Rules

Good passwords should:

Be long
Use multiple characters
Avoid common words
Not be reused

HTTPS

HTTPS encrypts communication between:

Browser

 |

Server

Protects:

Passwords
Personal data
API requests

Security Headers

Web applications use headers for protection.

Examples:

Content Security Policy
X-Frame-Options
HSTS

Common Security Tools

Bandit

Checks Python code for security issues.

Install:

pip install bandit

Run:

bandit app.py

Security Best Practices

1. Validate Input

if not username:
    return "Invalid"

2. Use HTTPS

Never send sensitive data over HTTP.

3. Update Dependencies

Keep libraries updated.

Command:

pip list --outdated

4. Hide Secrets

Use:

Environment variables
Secret managers

5. Limit User Permissions

Give users only required access.

Real Security Projects

1. Secure Login System

Features:

Registration
Password hashing
JWT authentication

2. Secure REST API

Features:

API keys
Authentication
Authorization

3. Password Manager

Features:

Encryption
Secure storage

Practice Exercises

Create password hashing system.
Build user registration.
Create JWT authentication.
Protect API routes.
Store secrets using environment variables.
Encrypt and decrypt messages.

Chapter 35: Python Cloud Deployment and DevOps Basics

What is Deployment?

Deployment means making your Python application available for users on the internet.

Example:

Development:

Your Computer
     |
     |
Python App

Deployment:

Users
  |
  |
Internet
  |
  |
Cloud Server
  |
  |
Python App

Development vs Production

Development Environment

Used by developers.

Features:

Debug mode ON
Local computer
Testing

Example:

localhost:8000

Production Environment

Used by real users.

Features:

Secure
Fast
Reliable
Monitored

Example:

example.com

What is DevOps?

DevOps combines:

Development
Operations

Goal:

Build, test, and deploy software faster.

DevOps Workflow

Write Code

    |

Test Code

    |

Build Application

    |

Deploy

    |

Monitor

Linux Basics for Python Developers

Most cloud servers use Linux.

Common commands:

Check Current Directory

pwd

List Files

ls

Change Directory

cd folder_name

Example:

cd project

Create Folder

mkdir app

Create File

touch main.py

Remove File

rm file.py

Installing Python on Server

Check Python:

python --version

Install packages:

pip install -r requirements.txt

Environment Variables in Production

Never store secrets in code.

Example:

Bad:

DATABASE_PASSWORD="12345"

Good:

import os

password=os.getenv(
"DATABASE_PASSWORD"
)

Hosting Options for Python

Popular cloud platforms:

AWS
Google Cloud
Microsoft Azure
DigitalOcean
Render
Railway

Types of Cloud Services

1. IaaS

Infrastructure as a Service.

You manage:

Server
Operating system
Software

Example:

Virtual machines.

2. PaaS

Platform as a Service.

Provider manages:

Server
Runtime
Deployment tools

Example:

Deploy Python app easily.

3. SaaS

Software as a Service.

Users directly use applications.

Examples:

Gmail
Online tools

Deploying Flask Application

Example structure:

project/

│

├── app.py

├── requirements.txt

└── templates/

requirements.txt

Contains dependencies:

Example:

flask
gunicorn

Create:

pip freeze > requirements.txt

Gunicorn

Flask’s built-in server is for development only.

Production uses:

Gunicorn

Install:

pip install gunicorn

Run:

gunicorn app:app

Deploying FastAPI

FastAPI commonly uses:

Uvicorn

Install:

pip install uvicorn

Run:

uvicorn main:app

Production:

uvicorn main:app --host 0.0.0.0

What is Docker?

Docker packages an application with everything needed to run.

Includes:

Python
Libraries
Code
Configuration

Without Docker

Computer A

Python 3.10

Library version 1


Computer B

Python 3.12

Different libraries

Problems:

Errors
Compatibility issues

With Docker

Docker Container

Python

Libraries

Application

Runs the same everywhere.

Installing Docker

Download from:

Docker Official Website

Docker Concepts

Image

A blueprint.

Example:

Python App Image

Container

Running instance of an image.

Example:

Running Python Application

Creating Dockerfile

File:

Dockerfile

Example:

FROM python:3.12

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD ["python","app.py"]

Building Docker Image

Command:

docker build -t myapp .

Running Container

Command:

docker run myapp

Docker Commands

View containers:

docker ps

Stop container:

docker stop container_id

Remove container:

docker rm container_id

Docker Compose

Used for multiple services.

Example:

Application:

Python App

     |

Database

     |

Redis

Docker Compose manages all.

File:

docker-compose.yml

Example Docker Compose

version: "3"

services:

  app:

    build: .

    ports:

      - "8000:8000"


  database:

    image: postgres

CI/CD Basics

CI/CD means:

Continuous Integration

Automatically:

Test code
Check errors

Continuous Deployment

Automatically:

Deploy application

CI/CD Pipeline

Developer Pushes Code

          |

       GitHub

          |

      Run Tests

          |

      Build App

          |

      Deploy

GitHub Actions

Used for automation.

Example:

.github/

   workflows/

       deploy.yml

Testing Before Deployment

Example:

pytest

If tests pass:

Deploy

If tests fail:

Stop Deployment

Web Servers

Production applications use web servers.

Popular:

Nginx

Used for:

Handling requests
Load balancing
Security

Architecture:

User

 |

Nginx

 |

Gunicorn/Uvicorn

 |

Python Application

Domain and DNS

Domain:

Example:

mywebsite.com

DNS connects:

Domain

   |

Server IP Address

SSL Certificate

SSL provides HTTPS.

Example:

http://

becomes

https://

Benefits:

Encryption
Security
User trust

Monitoring Applications

Production apps need monitoring.

Track:

Errors
CPU usage
Memory
Response time

Tools:

Prometheus
Grafana
Sentry

Logging in Production

Example:

import logging


logging.info(
"User logged in"
)

Logs help find problems.

Scaling Applications

When users increase:

100 users

     ↓

100000 users

Need scaling.

Vertical Scaling

Increase server power.

Example:

2GB RAM

↓

16GB RAM

Horizontal Scaling

Add more servers.

Example:

Server 1

Server 2

Server 3

Load Balancer

Distributes traffic.

Example:

Users

 |

Load Balancer

 |

----------------

|      |       |

App1  App2   App3

Cloud Deployment Projects

1. Deploy Flask Website

Steps:

Create app
Create requirements.txt
Configure server
Deploy

2. Deploy FastAPI API

Steps:

Build API
Dockerize
Deploy
Monitor

3. Full Production System

Components:

Frontend

    |

API

    |

Database

    |

Cloud Server

Practice Exercises

Deploy a Flask application.
Create a Docker image.
Run a Python app in Docker.
Create CI/CD workflow.
Deploy FastAPI API.
Configure environment variables.

Chapter 36: Python Advanced Concepts

Python has many powerful features beyond basic programming. These concepts help you write cleaner, faster, and professional-level Python code.

1. Iterators in Python

What is an Iterator?

An iterator is an object that allows you to access elements one by one.

Example:

numbers = [10,20,30]

for n in numbers:
    print(n)

Python internally uses an iterator.

Creating an Iterator

Use:

iter()

Example:

numbers = [10,20,30]

iterator = iter(numbers)


print(next(iterator))
print(next(iterator))
print(next(iterator))

Output:

10
20
30

next() Function

next() gets the next item from an iterator.

Example:

x = iter([1,2,3])

print(next(x))

Output:

Iterator Protocol

An object becomes an iterator when it has:

1. iter()

Returns iterator object.

2. next()

Returns next value.

Example:

class Counter:

    def __init__(self,max):

        self.max=max

        self.current=0


    def __iter__(self):

        return self


    def __next__(self):

        if self.current < self.max:

            value=self.current

            self.current += 1

            return value

        else:

            raise StopIteration

Usage:

for i in Counter(5):

    print(i)

Output:

2. Generators

What is a Generator?

A generator is a simple way to create iterators.

It uses:

yield

instead of:

return

Normal Function

Example:

def numbers():

    return [1,2,3]

Problem:

Stores everything in memory.

Generator Function

Example:

def numbers():

    yield 1

    yield 2

    yield 3

Usage:

for n in numbers():

    print(n)

Output:

1
2
3

Why Use Generators?

Advantages:

Saves memory
Faster for large data
Works with streams

Example:

Large file:

10 GB file

Generator:

Read one line at a time

Generator Expression

Similar to list comprehension.

List:

numbers=[x*x for x in range(10)]

Generator:

numbers=(x*x for x in range(10))

3. Decorators

What is a Decorator?

A decorator modifies or extends a function without changing its code.

Example:

Function

     |

Decorator

     |

Modified Function

Functions are Objects

Python treats functions as objects.

Example:

def hello():

    print("Hello")


x=hello

x()

Output:

Hello

Creating a Decorator

Example:

def decorator(func):

    def wrapper():

        print("Before function")

        func()

        print("After function")


    return wrapper

Use:

@decorator
def hello():

    print("Hello")


hello()

Output:

Before function
Hello
After function

Real Uses of Decorators

Used for:

Logging
Authentication
Performance measurement
Permission checking

Example: Login Decorator

def login_required(func):

    def wrapper(user):

        if user=="admin":

            return func(user)

        else:

            return "Access denied"


    return wrapper

4. Context Managers

What is a Context Manager?

Used to manage resources automatically.

Examples:

Files
Database connections
Network connections

Without Context Manager

file=open(
"data.txt"
)

data=file.read()

file.close()

Problem:

If error happens:

File may remain open

With Context Manager

Use:

with

Example:

with open("data.txt") as file:

    data=file.read()

Python automatically closes the file.

Creating Custom Context Manager

Using class:

class MyFile:

    def __enter__(self):

        print("Opening")


    def __exit__(self,exc_type,exc,value):

        print("Closing")

Usage:

with MyFile():

    print("Working")

5. Lambda Functions

What is Lambda?

A small anonymous function.

Normal:

def add(a,b):

    return a+b

Lambda:

add=lambda a,b:a+b

Usage:

print(add(5,3))

Output:

Lambda with map()

Example:

numbers=[1,2,3,4]


result=list(
map(
lambda x:x*2,
numbers
)
)


print(result)

Output:

[2,4,6,8]

Lambda with filter()

Example:

numbers=[1,2,3,4,5]


even=list(
filter(
lambda x:x%2==0,
numbers
)
)


print(even)

Output:

[2,4]

6. Advanced Object-Oriented Programming

Multiple Inheritance

A class can inherit from multiple classes.

Example:

class A:

    def show(self):

        print("A")


class B:

    def display(self):

        print("B")


class C(A,B):

    pass

Method Resolution Order (MRO)

Python decides which method to call.

Example:

print(C.mro())

Abstract Classes

Abstract classes define rules for child classes.

Use:

abc

module.

Example:

from abc import ABC,abstractmethod


class Animal(ABC):


    @abstractmethod

    def sound(self):

        pass

Static Methods

A method that does not use object data.

Example:

class Math:

    @staticmethod

    def add(a,b):

        return a+b

Usage:

Math.add(2,3)

Class Methods

Works with class itself.

Example:

class Student:

    school="ABC"


    @classmethod

    def show_school(cls):

        print(cls.school)

7. Python Memory Management

Python automatically manages memory.

It uses:

Reference counting
Garbage collection

Reference Counting

Example:

a=[1,2,3]

b=a

Now:

Two references

Garbage Collection

Unused objects are removed automatically.

Example:

import gc

gc.collect()

8. Shallow Copy and Deep Copy

Shallow Copy

Copies object structure but shares inner objects.

Example:

import copy


a=[[1,2]]

b=copy.copy(a)

Deep Copy

Creates completely independent copy.

Example:

b=copy.deepcopy(a)

9. Type Hints

Type hints improve readability.

Example:

Without:

def add(a,b):

    return a+b

With:

def add(
a:int,
b:int
)->int:

    return a+b

10. Dataclasses

Used to create data classes easily.

Example:

from dataclasses import dataclass


@dataclass

class User:

    name:str

    age:int

Usage:

u=User(
"John",
20
)

11. Regular Expressions (Regex)

Used for searching patterns.

Library:

import re

Finding Text

Example:

import re


text="My phone is 12345"


result=re.findall(
"\d+",
text
)


print(result)

Output:

['12345']

Regex Uses

Email validation
Password checking
Data extraction
Text processing

Advanced Python Projects

1. Web Framework Components

Uses:

Decorators
Context managers
Classes

2. Data Processing Engine

Uses:

Generators
Iterators

3. Authentication System

Uses:

Decorators
Encryption
Classes

Practice Exercises

Create your own iterator.
Build a generator for large files.
Create logging decorator.
Create a custom context manager.
Use lambda with map/filter.
Build classes using advanced OOP.
Practice regex patterns.

Chapter 37: Python Internals and Performance Optimization

Python is easy to use, but professional developers should understand how Python works internally and how to make programs faster.

1. How Python Executes Code

When you run:

print("Hello Python")

Python goes through several steps:

Python Code (.py)

        |

Python Interpreter

        |

Bytecode

        |

Python Virtual Machine

        |

Machine Execution

2. Python Interpreter

The interpreter reads and executes Python code.

Popular Python implementations:

CPython

The default and most common implementation.

Written in:

C Language

PyPy

A faster Python implementation using JIT compilation.

Jython

Python running on Java Virtual Machine.

IronPython

Python running on .NET.

3. Python Bytecode

Python does not directly execute source code.

Example:

File:

x = 10
print(x)

Converted into:

Bytecode

Python Virtual Machine executes bytecode.

Viewing Bytecode

Python provides:

dis

module.

Example:

import dis


def hello():

    print("Hello")


dis.dis(hello)

Output:

LOAD_GLOBAL
CALL
RETURN

4. Python Memory Model

Python stores objects in memory.

Example:

x = 100

Memory:

Variable

  |

Object

  |

100

Variables are References

Example:

a = [1,2,3]

b = a

Both point to the same object:

a ----\
       \
        [1,2,3]
       /
b ----/

Checking Object Identity

Use:

id()

Example:

a=10

print(id(a))

5. Garbage Collection

Python automatically removes unused objects.

Example:

a=[1,2,3]

del a

Memory can be released.

Garbage Collector Module

import gc


gc.collect()

6. Python Global Interpreter Lock (GIL)

What is GIL?

GIL is a mechanism in CPython that allows only one thread to execute Python bytecode at a time.

Example:

Thread 1  ----\
              \
               Python Interpreter
              /
Thread 2  ----/

Why Does GIL Exist?

It protects Python memory management.

Benefits:

Simpler interpreter
Safer object handling

GIL Problem

For CPU-heavy tasks:

Example:

Image Processing
Machine Learning
Large Calculations

Threads may not improve speed.

7. Multithreading

What is Threading?

Running multiple tasks using threads.

Useful for:

Network requests
File operations
Waiting tasks

Thread Example

import threading


def task():

    print("Running task")


thread = threading.Thread(
target=task
)


thread.start()

thread.join()

Multiple Threads

Example:

import threading


def work(number):

    print(number)


for i in range(5):

    t=threading.Thread(
    target=work,
    args=(i,)
    )

    t.start()

Threading Limitations

Good for:

✅ I/O tasks

Not good for:

❌ Heavy calculations

8. Multiprocessing

Multiprocessing creates separate Python processes.

Each process has:

Separate memory
Separate interpreter

Structure:

Process 1

Python Interpreter


Process 2

Python Interpreter

Multiprocessing Example

from multiprocessing import Process


def task():

    print("Process running")


p=Process(
target=task
)


p.start()

p.join()

Threading vs Multiprocessing

Threading	Multiprocessing
Same memory	Separate memory
Lightweight	More resources
Good for I/O	Good for CPU
Affected by GIL	Avoids GIL

9. Async Programming

What is Async?

Async allows Python to handle many tasks without waiting.

Example:

Normal:

Task A
(wait)
Task B
(wait)
Task C

Async:

Task A starts

while waiting:

Task B runs

Task C runs

asyncio Module

Python provides:

asyncio

Async Function

Use:

async def

Example:

import asyncio


async def hello():

    print("Hello")


asyncio.run(
hello()
)

Await Keyword

Used to wait for async operation.

Example:

import asyncio


async def task():

    await asyncio.sleep(2)

    print("Done")

Running Multiple Async Tasks

Example:

import asyncio


async def work(number):

    print(number)


async def main():

    await asyncio.gather(

        work(1),

        work(2),

        work(3)

    )


asyncio.run(main())

10. Profiling Python Code

Profiling finds slow parts of programs.

time Module

Example:

import time


start=time.time()


# code


end=time.time()


print(
end-start
)

cProfile

Built-in profiler.

Example:

python -m cProfile app.py

11. Optimization Techniques

1. Use Efficient Data Structures

Slow:

list

For searching:

set

is faster.

Example:

names=set(
["John","Alex"]
)

print(
"John" in names
)

2. Avoid Repeated Calculations

Bad:

for i in range(1000):

    result=calculate()

Better:

result=calculate()

for i in range(1000):

    print(result)

3. Use List Comprehension

Slow:

result=[]

for x in range(10):

    result.append(x*x)

Better:

result=[
x*x for x in range(10)
]

4. Use Generators for Large Data

List:

numbers=[
x for x in range(1000000)
]

Uses large memory.

Generator:

numbers=(
x for x in range(1000000)
)

Uses less memory.

5. Use Caching

Caching stores previous results.

Example:

from functools import lru_cache


@lru_cache
def square(x):

    return x*x

12. Memory Optimization

Check Object Size

Use:

import sys


print(
sys.getsizeof(100)
)

13. Database Optimization

Avoid:

1000 database queries

Better:

1 optimized query

14. Async Web Applications

Frameworks supporting async:

FastAPI
aiohttp

Example:

async def get_data():

    data = await request()

    return data

15. Python Performance Tools

Tools:

cProfile

Code profiling

timeit

Benchmark small code.

Example:

import timeit


print(
timeit.timeit(
"sum(range(100))"
)
)

memory_profiler

Tracks memory usage.

Real Performance Projects

1. Fast API Server

Uses:

Async
Database optimization
Caching

2. Data Processing System

Uses:

Multiprocessing
Generators

3. Machine Learning Pipeline

Uses:

Parallel processing
Memory optimization

Practice Exercises

View Python bytecode using dis.
Create a multithreading program.
Create a multiprocessing program.
Build an async program.
Profile slow code.
Optimize a large data-processing script.

Chapter 38: Python Web Development Complete Guide

What is Web Development?

Web Development is the process of creating websites and web applications that run through browsers.

Examples:

Online shopping websites
Social media platforms
Banking applications
APIs
Dashboards

How the Web Works

When a user opens a website:

User Browser

      |

      | HTTP Request

      ↓

Web Server

      |

      | Process Request

      ↓

Python Application

      |

      | Response

      ↓

User Browser

Frontend vs Backend

Frontend

The part users see.

Technologies:

HTML
CSS
JavaScript

Examples:

Buttons
Forms
Pages
Designs

Backend

The server-side logic.

Python handles:

Business logic
Database operations
Authentication
APIs

Full Stack Architecture

Frontend

(HTML/CSS/JS)

       |

       |

Backend

(Python Flask/FastAPI)

       |

       |

Database

(MySQL/PostgreSQL)

HTTP Basics

HTTP means:

HyperText Transfer Protocol

It allows communication between browser and server.

HTTP Request

Example:

GET /users

Means:

“Give me users data”

HTTP Methods

GET

Retrieve data.

Example:

GET /products

POST

Create new data.

Example:

POST /users

PUT

Update data.

Example:

PUT /profile

DELETE

Remove data.

Example:

DELETE /user/5

HTTP Status Codes

Success

200

Request successful.

201

Created successfully.

Client Errors

400

Bad request.

401

Unauthorized.

404

Not found.

Server Errors

500

Server error.

Python Web Frameworks

Popular frameworks:

Flask

Simple and lightweight.

Good for:

Small applications
APIs
Learning

Django

Full-featured framework.

Includes:

Authentication
Admin panel
ORM
Security

FastAPI

Modern API framework.

Features:

High performance
Async support
Automatic documentation

Flask Framework

Install:

pip install flask

First Flask Application

Create:

app.py

Code:

from flask import Flask


app = Flask(__name__)


@app.route("/")
def home():

    return "Hello Python Web"


app.run()

Running Flask

Command:

python app.py

Open:

http://127.0.0.1:5000

Flask Routing

Routes connect URLs with functions.

Example:

@app.route("/about")
def about():

    return "About Page"

URL:

/about

Dynamic Routes

Example:

@app.route("/user/<name>")
def user(name):

    return name

URL:

/user/John

Output:

John

HTTP Methods in Flask

Example:

from flask import request


@app.route(
"/login",
methods=["POST"]
)

def login():

    data=request.json

    return data

Flask Templates

Web pages are created using HTML templates.

Structure:

project/

|

├── app.py

|

└── templates/

      index.html

Rendering HTML

Example:

from flask import render_template


@app.route("/")
def home():

    return render_template(
    "index.html"
    )

Template Variables

HTML:

<h1>
{{name}}
</h1>

Python:

return render_template(
"index.html",
name="John"
)

Static Files

Used for:

CSS
JavaScript
Images

Structure:

project/

├── static/

│     ├── style.css

│     └── image.png

Flask Forms

Example:

HTML:

<form method="POST">

<input name="username">

<button>
Submit
</button>

</form>

Python:

username=request.form["username"]

Flask Database Connection

Common:

SQLite
MySQL
PostgreSQL

Using SQLAlchemy:

Install:

pip install flask-sqlalchemy

Flask API Example

from flask import Flask,jsonify


app=Flask(__name__)


@app.route("/api/users")

def users():

    data=[
        {
        "name":"John"
        }
    ]

    return jsonify(data)

Response:

[
{
"name":"John"
}
]

FastAPI Framework

Install:

pip install fastapi uvicorn

First FastAPI App

Create:

main.py

Code:

from fastapi import FastAPI


app=FastAPI()


@app.get("/")
def home():

    return {
    "message":"Hello FastAPI"
    }

Running FastAPI

Command:

uvicorn main:app --reload

FastAPI Routes

GET:

@app.get("/users")
def users():

    return ["John","Alex"]

POST:

@app.post("/users")
def create_user():

    return {
    "status":"created"
    }

FastAPI Path Parameters

Example:

@app.get("/user/{id}")

def user(id:int):

    return {
    "id":id
    }

Request Body with Pydantic

Pydantic validates data.

Example:

from pydantic import BaseModel


class User(BaseModel):

    name:str

    age:int

Using model:

@app.post("/users")

def create(user:User):

    return user

Automatic API Documentation

FastAPI automatically creates:

Swagger UI:

/docs

ReDoc:

/redoc

REST API Design

REST means:

Representational State Transfer.

Example:

Users API:

GET /users

POST /users

GET /users/1

PUT /users/1

DELETE /users/1

Authentication in Web Apps

Common methods:

Session Authentication

Used for websites.

Example:

Login

↓

Session Created

↓

Access Pages

JWT Authentication

Used for APIs.

Example:

Login

↓

JWT Token

↓

API Access

Connecting Frontend and Backend

Example:

React Frontend

        |

        |

FastAPI Backend

        |

        |

Database

Communication:

HTTP requests
JSON data

CORS

CORS allows frontend and backend to communicate.

Install:

pip install flask-cors

Example:

from flask_cors import CORS

CORS(app)

Web Application Security

Important:

1. Validate Input

Never trust user data.

2. Hash Passwords

Use:

bcrypt

3. Use HTTPS

Encrypt communication.

4. Protect APIs

Use:

JWT
API keys

Testing Web Applications

Tools:

Pytest

Testing framework.

Install:

pip install pytest

Example:

def test_home():

    assert 1+1==2

Deployment Architecture

Production example:

User

 |

Nginx

 |

Gunicorn/Uvicorn

 |

Flask/FastAPI

 |

Database

Real Web Development Projects

1. Blog Website

Features:

Users
Posts
Comments
Authentication

2. E-Commerce Website

Features:

Products
Cart
Orders
Payments

3. REST API Service

Features:

User API
Database
Authentication

Practice Exercises

Create a Flask website.
Build REST API.
Connect database.
Add login system.
Create FastAPI project.
Deploy web application.

Chapter 39: Django Framework Complete Guide

What is Django?

Django is a powerful Python web framework used to build secure and scalable web applications.

Django follows:

MVT Architecture

MVT means:

Model
View
Template

Why Use Django?

Django provides built-in features:

✅ Database ORM
✅ User authentication
✅ Admin panel
✅ Security protection
✅ URL routing
✅ Form handling
✅ Session management

Used by many large websites.

Django Architecture (MVT)

User Browser

      |

      |

     URL

      |

      |

    View

      |

      |

   Model

      |

      |

 Database


      |

      |

 Template

      |

      |

 HTML Response

Installing Django

Install:

pip install django

Check version:

django-admin --version

Creating a Django Project

Command:

django-admin startproject myproject

Structure:

myproject/

│

├── manage.py

│

└── myproject/

    ├── settings.py

    ├── urls.py

    ├── wsgi.py

    └── asgi.py

Running Django Server

Go inside project:

cd myproject

Run:

python manage.py runserver

Open:

http://127.0.0.1:8000

Django Project Files

manage.py

Used to manage project.

Examples:

python manage.py runserver

python manage.py migrate

settings.py

Contains:

Database settings
Installed apps
Security settings

urls.py

Controls website URLs.

Example:

urlpatterns = []

Creating a Django App

A Django project contains multiple apps.

Create:

python manage.py startapp blog

Structure:

blog/

├── models.py

├── views.py

├── admin.py

├── apps.py

└── migrations/

Project vs App

Project	App
Complete website	Feature/module
Settings	Functionality
Contains apps	Contains logic

Example:

E-commerce Project

    |

    ├── Users App

    ├── Products App

    └── Orders App

Adding App to Django

Open:

settings.py

Add:

INSTALLED_APPS=[

'blog',

]

Django Models

Models define database tables.

Example:

from django.db import models


class Post(models.Model):

    title=models.CharField(
    max_length=100
    )

    content=models.TextField()

    created=models.DateTimeField(
    auto_now_add=True
    )

Creating Database Tables

Make migrations:

python manage.py makemigrations

Apply:

python manage.py migrate

Django ORM

ORM allows database operations using Python.

Instead of SQL:

SELECT * FROM post;

Use:

Post.objects.all()

Creating Data

Example:

post=Post(

title="Python",

content="Learning Django"

)


post.save()

Reading Data

All records:

Post.objects.all()

Filter:

Post.objects.filter(
title="Python"
)

Get one:

Post.objects.get(
id=1
)

Updating Data

Example:

post=Post.objects.get(id=1)

post.title="Django"

post.save()

Deleting Data

Example:

post.delete()

Django Views

Views handle requests and responses.

File:

views.py

Example:

from django.http import HttpResponse


def home(request):

    return HttpResponse(
    "Hello Django"
    )

Django URLs

Connect URL to view.

urls.py:

from django.urls import path

from . import views


urlpatterns=[

path(
"",
views.home
)

]

Templates

Templates create HTML pages.

Folder:

templates/

    home.html

Rendering Templates

View:

from django.shortcuts import render


def home(request):

    return render(
    request,
    "home.html"
    )

Template Variables

HTML:

<h1>
{{name}}
</h1>

View:

return render(

request,

"home.html",

{
"name":"John"
}

)

Output:

John

Template Conditions

Example:

{% if user %}

Welcome

{% else %}

Login

{% endif %}

Template Loops

Example:

{% for item in items %}

<p>
{{item}}
</p>

{% endfor %}

Django Admin Panel

Django provides automatic admin interface.

Create admin user:

python manage.py createsuperuser

Enter:

Username
Email
Password

Register Models in Admin

admin.py:

from django.contrib import admin

from .models import Post


admin.site.register(Post)

Access:

/admin

Django Forms

Forms collect user input.

Example:

from django import forms


class PostForm(forms.Form):

    title=forms.CharField()

    content=forms.CharField()

Form Processing

Example:

if request.method=="POST":

    form=PostForm(
    request.POST
    )

    if form.is_valid():

        print(
        form.cleaned_data
        )

Django Authentication

Django includes:

Login
Logout
Password reset
User management

Creating Login View

Example:

from django.contrib.auth import authenticate


user=authenticate(

username="john",

password="pass"

)

User Model

Built-in:

User

Contains:

Username
Password
Email
Permissions

Sessions in Django

Store user information.

Example:

request.session["name"]="John"

Django Middleware

Middleware processes requests before views.

Examples:

Authentication
Security
Logging

Flow:

Request

 |

Middleware

 |

View

 |

Response

Django Static Files

Used for:

CSS
JavaScript
Images

Structure:

static/

├── style.css

└── image.png

Django Security Features

Built-in protection:

CSRF Protection

Protects forms.

SQL Injection Protection

ORM prevents unsafe queries.

XSS Protection

Protects against malicious scripts.

Clickjacking Protection

Prevents UI attacks.

Django REST Framework (DRF)

Used to create APIs with Django.

Install:

pip install djangorestframework

Creating API

Example:

from rest_framework.views import APIView

from rest_framework.response import Response


class UserAPI(APIView):

    def get(self,request):

        return Response(
        {
        "name":"John"
        }
        )

Django Project Structure Example

Ecommerce/

|

├── users/

|

├── products/

|

├── orders/

|

├── templates/

|

├── static/

|

└── manage.py

Real Django Projects

1. Blog Website

Features:

Posts
Comments
Users

2. E-commerce Website

Features:

Products
Cart
Orders
Payments

3. Social Media App

Features:

Profiles
Posts
Likes
Messages

Practice Exercises

Install Django.
Create a project.
Create an app.
Create models.
Connect database.
Build templates.
Create authentication.
Build REST API.

Chapter 40: Python API Development Masterclass

What is an API?

API means:

Application Programming Interface

An API allows different applications to communicate.

Example:

Mobile App

      |

      | API Request

      ↓

Python Backend

      |

      | Database

      ↓

API Response

      |

      ↓

Mobile App

Why Use APIs?

APIs are used for:

Mobile applications
Web applications
Payment systems
Third-party integrations
Microservices

Types of APIs

1. REST API

Most common.

Uses:

HTTP
JSON

Example:

GET /users

2. SOAP API

Uses XML.

Common in:

Banking
Enterprise systems

3. GraphQL API

Allows clients to request specific data.

Example:

{
 user {
   name
 }
}

REST API Principles

REST follows:

1. Resources

Everything is a resource.

Example:

/users

/products

/orders

2. HTTP Methods

Method	Purpose
GET	Read
POST	Create
PUT	Update
DELETE	Remove

API Request and Response

Request:

GET /users/1

Response:

{
"id":1,
"name":"John"
}

JSON Data Format

JSON means:

JavaScript Object Notation

Example:

{
"name":"Alice",
"age":25,
"city":"Delhi"
}

Python equivalent:

data={
"name":"Alice",
"age":25
}

Building REST APIs with FastAPI

Install:

pip install fastapi uvicorn

Basic FastAPI Application

File:

main.py

Code:

from fastapi import FastAPI


app=FastAPI()


@app.get("/")
def home():

    return {
    "message":"API Running"
    }

Run:

uvicorn main:app --reload

Creating GET API

Example:

users=[
{
"id":1,
"name":"John"
}
]


@app.get("/users")
def get_users():

    return users

Response:

[
{
"id":1,
"name":"John"
}
]

Path Parameters

Used to get specific data.

Example:

@app.get("/users/{id}")

def get_user(id:int):

    return {
    "user_id":id
    }

Request:

/users/5

Response:

{
"user_id":5
}

Query Parameters

Used for filtering.

Example:

@app.get("/products")

def products(
category:str
):

    return {
    "category":category
    }

Request:

/products?category=mobile

POST API

Used to create data.

Example:

from pydantic import BaseModel


class User(BaseModel):

    name:str

    age:int



@app.post("/users")

def create_user(user:User):

    return user

Request:

{
"name":"John",
"age":20
}

PUT API

Update existing data.

Example:

@app.put("/users/{id}")

def update_user(
id:int,
user:User
):

    return {
    "id":id,
    "data":user
    }

DELETE API

Example:

@app.delete("/users/{id}")

def delete_user(id:int):

    return {
    "deleted":id
    }

API Validation with Pydantic

Pydantic checks input data.

Example:

from pydantic import BaseModel


class Product(BaseModel):

    name:str

    price:float

    quantity:int

Invalid:

{
"name":100
}

FastAPI rejects it.

API Response Models

Control output format.

Example:

@app.get(
"/user",
response_model=User
)

def user():

    return data

HTTP Status Codes in FastAPI

Example:

from fastapi import status


@app.post(
"/users",
status_code=status.HTTP_201_CREATED
)

Error Handling

Example:

from fastapi import HTTPException


@app.get("/users/{id}")

def user(id:int):

    if id!=1:

        raise HTTPException(
        status_code=404,
        detail="User not found"
        )

    return {
    "name":"John"
    }

Connecting API with Database

Architecture:

Client

 |

FastAPI

 |

SQLAlchemy ORM

 |

Database

SQLAlchemy Installation

pip install sqlalchemy

Database Model Example

from sqlalchemy import Column,Integer,String


class User:

    id=Column(
    Integer,
    primary_key=True
    )

    name=Column(String)

API Authentication

APIs need protection.

Common methods:

JWT
OAuth2
API Keys

JWT Authentication Flow

User Login

    |

Verify Password

    |

Generate JWT Token

    |

Client Stores Token

    |

Send Token With Requests

    |

Access Granted

OAuth2

Used by:

Google Login
Facebook Login
Enterprise apps

Flow:

User

 |

Login Provider

 |

Authorization Code

 |

Access Token

 |

Application

API Key Authentication

Example:

Request:

GET /data

Authorization:
API-Key abc123

Middleware in FastAPI

Middleware runs before every request.

Example:

@app.middleware("http")

async def middleware(
request,
call_next
):

    response=await call_next(request)

    return response

Background Tasks

Run tasks after response.

Examples:

Sending emails
Processing files
Notifications

Example:

from fastapi import BackgroundTasks


@app.post("/send")

def send(
background_tasks:BackgroundTasks
):

    background_tasks.add_task(
    send_email
    )

    return {
    "status":"started"
    }

File Upload API

Example:

from fastapi import UploadFile


@app.post("/upload")

def upload(
file:UploadFile
):

    return {
    "filename":file.filename
    }

API Documentation

FastAPI automatically provides:

Swagger:

/docs

ReDoc:

/redoc

Testing APIs

Tools:

Postman

Used for:

Sending requests
Testing responses

Pytest

Example:

def test_api():

    assert True

API Security

Important:

Validate Input

Prevent bad data.

Rate Limiting

Prevent abuse.

Example:

100 requests/minute

HTTPS

Encrypt communication.

CORS

Control allowed websites.

API Versioning

Large APIs need versions.

Example:

Version 1:

/api/v1/users

Version 2:

/api/v2/users

Pagination

Used for large data.

Example:

Instead of:

100000 users

Return:

Page 1
20 users

Example:

/users?page=1&limit=20

Caching

Stores frequently used data.

Tools:

Redis
Memcached

Example:

Database

   |

Cache

   |

API Response

Logging API Requests

Track:

User activity
Errors
Performance

Example:

import logging


logging.info(
"API called"
)

Production API Architecture

Client Apps

      |

Load Balancer

      |

Nginx

      |

FastAPI Servers

      |

Database

      |

Redis Cache

Real API Projects

1. Banking API

Features:

User accounts
Transactions
Security

2. E-commerce API

Features:

Products
Orders
Payments

3. Social Media API

Features:

Users
Posts
Messages

Practice Exercises

Create CRUD API.
Add database connection.
Add JWT login.
Add API documentation.
Add file upload.
Deploy API.
Add testing.

Chapter 41: Python Data Science and Machine Learning Introduction

What is Data Science?

Data Science is the process of collecting, analyzing, and extracting useful information from data.

It combines:

Programming
Mathematics
Statistics
Machine Learning
Data Visualization

Example:

Raw Data

   |

Data Processing

   |

Analysis

   |

Useful Information

   |

Decision Making

Why Learn Data Science with Python?

Python is popular because it has powerful libraries:

Library	Purpose
NumPy	Numerical computing
Pandas	Data analysis
Matplotlib	Visualization
Seaborn	Statistical graphs
Scikit-learn	Machine Learning
TensorFlow	Deep Learning
PyTorch	AI models

Data Science Workflow

A typical project follows:

1. Collect Data

        |

2. Clean Data

        |

3. Explore Data

        |

4. Visualize Data

        |

5. Build Model

        |

6. Test Model

        |

7. Deploy

Types of Data

1. Structured Data

Data stored in tables.

Example:

Name	Age	Salary
John	25	50000
Alex	30	70000

Examples:

Excel files
Databases
CSV files

2. Unstructured Data

No fixed format.

Examples:

Images
Videos
Audio
Text documents

3. Semi-Structured Data

Partially organized.

Examples:

JSON
XML

Example:

{
"name":"John",
"age":25
}

Installing Data Science Libraries

Install using pip:

pip install numpy pandas matplotlib seaborn scikit-learn

1. NumPy Introduction

What is NumPy?

NumPy means:

Numerical Python

It is used for:

Arrays
Mathematical calculations
Scientific computing

Import NumPy

import numpy as np

Creating NumPy Array

Python list:

numbers=[1,2,3,4]

NumPy array:

import numpy as np


arr=np.array(
[1,2,3,4]
)


print(arr)

Output:

[1 2 3 4]

Why NumPy Arrays?

Compared with Python lists:

Advantages:

Faster calculations
Less memory usage
Supports mathematical operations

Array Dimensions

1D Array

arr=np.array(
[1,2,3]
)

Shape:

(3,)

2D Array

Matrix:

arr=np.array(
[
[1,2,3],
[4,5,6]
]
)

Shape:

(2,3)

Checking Array Properties

Example:

print(arr.ndim)

Number of dimensions.

print(arr.shape)

Size of array.

print(arr.dtype)

Data type.

Creating Special Arrays

Zeros

np.zeros(5)

Output:

[0. 0. 0. 0. 0.]

Ones

np.ones(5)

Output:

[1. 1. 1. 1. 1.]

Range

np.arange(1,10)

Output:

[1 2 3 4 5 6 7 8 9]

Random Numbers

np.random.rand(5)

Example output:

0.54 0.23 0.88

NumPy Mathematical Operations

Array:

a=np.array(
[1,2,3]
)

Addition:

a+5

Output:

[6 7 8]

Multiplication:

a*2

Output:

[2 4 6]

Array Statistics

Mean

Average value.

np.mean(a)

Maximum

np.max(a)

Minimum

np.min(a)

Sum

np.sum(a)

Array Indexing

Example:

arr=np.array(
[10,20,30,40]
)

First element:

arr[0]

Output:

Last element:

arr[-1]

Output:

Array Slicing

Example:

arr[1:3]

Output:

[20 30]

2D Array Access

Example:

matrix=np.array(
[
[1,2],
[3,4]
]
)

Access:

matrix[0][1]

Output:

NumPy Matrix Operations

Example:

a=np.array(
[
[1,2],
[3,4]
]
)


b=np.array(
[
[5,6],
[7,8]
]
)

Addition:

a+b

Output:

[[6 8]
 [10 12]]

Dot Product

Used heavily in Machine Learning.

Example:

np.dot(a,b)

2. Pandas Introduction

What is Pandas?

Pandas is a library for:

Data analysis
Data cleaning
Data manipulation

It works with:

CSV files
Excel files
Databases

Import Pandas

import pandas as pd

Pandas Series

A Series is a one-dimensional labeled array.

Example:

data=pd.Series(
[10,20,30]
)

print(data)

Output:

0    10
1    20
2    30

Pandas DataFrame

A DataFrame is a table.

Example:

data={
"Name":[
"John",
"Alice"
],

"Age":[20,25]

}


df=pd.DataFrame(data)


print(df)

Output:

Name    Age

John    20

Alice   25

Reading CSV Files

Example:

df=pd.read_csv(
"data.csv"
)

Viewing Data

First rows:

df.head()

Last rows:

df.tail()

Checking Information

df.info()

Shows:

Columns
Data types
Missing values

Statistics Summary

df.describe()

Shows:

Mean
Count
Min
Max

Selecting Columns

Example:

df["Age"]

Filtering Data

Example:

df[
df["Age"]>20
]

Adding New Column

Example:

df["Country"]="India"

Removing Column

Example:

df.drop(
"Country",
axis=1
)

Handling Missing Data

Check:

df.isnull()

Remove:

df.dropna()

Fill:

df.fillna(0)

Data Science Practice Project

Student Performance Analysis

Dataset:

Students.csv

Columns:

Name
Math
Science
English

Tasks:

Load data
Find average marks
Find highest scorer
Create graphs
Analyze performance

Chapter 42: Data Visualization with Python

What is Data Visualization?

Data Visualization is the process of representing data using charts, graphs, and visual elements.

Instead of looking at thousands of numbers:

We create:

Chart → Pattern → Understanding → Decision

Why Data Visualization is Important

It helps to:

Find patterns
Detect errors
Understand trends
Compare values
Present results

Examples:

Business reports
Scientific research
Machine learning analysis
Financial dashboards

Popular Python Visualization Libraries

Library	Purpose
Matplotlib	Basic plotting
Seaborn	Statistical visualization
Plotly	Interactive charts
Bokeh	Web-based visualization

1. Matplotlib Introduction

What is Matplotlib?

Matplotlib is the most popular Python visualization library.

Used for:

Line charts
Bar charts
Histograms
Scatter plots
Pie charts

Installing Matplotlib

pip install matplotlib

Import Matplotlib

import matplotlib.pyplot as plt

Creating First Graph

Example:

import matplotlib.pyplot as plt


x=[1,2,3,4]

y=[10,20,30,40]


plt.plot(x,y)


plt.show()

Output:

A line graph showing increasing values.

Adding Title

plt.title(
"Sales Growth"
)

Adding Labels

X-axis:

plt.xlabel(
"Months"
)

Y-axis:

plt.ylabel(
"Sales"
)

Complete Line Chart

import matplotlib.pyplot as plt


months=[
"Jan",
"Feb",
"Mar",
"Apr"
]


sales=[
100,
200,
300,
400
]


plt.plot(
months,
sales
)


plt.title(
"Monthly Sales"
)


plt.xlabel(
"Month"
)


plt.ylabel(
"Sales"
)


plt.show()

2. Bar Chart

Used for comparing categories.

Example:

import matplotlib.pyplot as plt


products=[
"Phone",
"Laptop",
"Tablet"
]


sales=[
50,
80,
30
]


plt.bar(
products,
sales
)


plt.title(
"Product Sales"
)


plt.show()

Real Uses of Bar Charts

Examples:

Product comparison
Company revenue
Student marks

3. Horizontal Bar Chart

Example:

plt.barh(
products,
sales
)

Useful when labels are long.

4. Scatter Plot

Shows relationship between two variables.

Example:

x=[
1,2,3,4,5
]


y=[
2,4,6,8,10
]


plt.scatter(
x,y
)


plt.show()

Uses of Scatter Plot

Examples:

Height vs Weight
Advertising vs Sales
Temperature vs Electricity usage

5. Histogram

Shows data distribution.

Example:

ages=[
18,20,22,25,30,35,40
]


plt.hist(
ages
)


plt.show()

Uses of Histogram

Age distribution
Exam scores
Data analysis

6. Pie Chart

Shows percentage distribution.

Example:

labels=[
"Python",
"Java",
"C++"
]


values=[
50,
30,
20
]


plt.pie(
values,
labels=labels
)


plt.show()

7. Multiple Graphs

Example:

plt.plot(
[1,2,3],
[10,20,30]
)


plt.plot(
[1,2,3],
[30,20,10]
)


plt.show()

8. Saving Charts

Example:

plt.savefig(
"chart.png"
)

9. Seaborn Introduction

What is Seaborn?

Seaborn is built on Matplotlib.

It provides:

Better styling
Statistical charts
Easy data analysis

Installing Seaborn

pip install seaborn

Import Seaborn

import seaborn as sns

Loading Sample Dataset

Example:

import seaborn as sns


data=sns.load_dataset(
"tips"
)


print(data.head())

Seaborn Scatter Plot

Example:

sns.scatterplot(

data=data,

x="total_bill",

y="tip"

)

Seaborn Bar Plot

Example:

sns.barplot(

data=data,

x="day",

y="total_bill"

)

Box Plot

Used to understand data spread.

Example:

sns.boxplot(

data=data,

x="day",

y="total_bill"

)

Heatmap

Used for correlation.

Example:

sns.heatmap(
data.corr()
)

Correlation

Shows relationship between variables.

Range:

-1  → Negative relationship

0   → No relationship

1   → Positive relationship

Example:

More study hours

        ↓

Higher marks

Positive correlation.

10. Data Visualization with Pandas

Pandas can create charts directly.

Example:

df.plot()

Line Chart with Pandas

df["Sales"].plot(
kind="line"
)

Bar Chart with Pandas

df.plot(
kind="bar"
)

11. Real Data Visualization Workflow

Example:

Sales Analysis:

CSV File

   |

Pandas

   |

Clean Data

   |

Matplotlib

   |

Charts

   |

Business Decision

Example Project: Company Sales Dashboard

Dataset:

sales.csv

Columns:

Date
Product
Region
Sales
Profit

Tasks:

Step 1: Load Data

df=pd.read_csv(
"sales.csv"
)

Step 2: Analyze Sales

df.describe()

Step 3: Monthly Sales Graph

plt.plot(
df["Date"],
df["Sales"]
)

Step 4: Product Comparison

plt.bar(
df["Product"],
df["Sales"]
)

12. Advanced Visualization Concepts

Subplots

Multiple charts in one figure.

Example:

plt.subplot(
1,
2,
1
)

Annotations

Adding notes:

plt.annotate(
"Highest",
xy=(3,400)
)

Legends

Explain chart lines.

Example:

plt.legend()

13. Interactive Visualization

Libraries:

Plotly

Install:

pip install plotly

Example:

import plotly.express as px


fig=px.line(
x=[1,2,3],
y=[10,20,30]
)


fig.show()

Real-World Visualization Projects

1. COVID Data Analysis

Charts:

Cases over time
Country comparison
Growth rate

2. Stock Market Analysis

Charts:

Price movement
Volume
Trends

3. Business Dashboard

Features:

Revenue charts
Customer analysis
Profit reports

Visualization Best Practices

1. Choose Correct Chart

Purpose	Chart
Compare values	Bar
Show trend	Line
Distribution	Histogram
Relationship	Scatter
Percentage	Pie

2. Avoid Too Much Information

Bad:

100 colors
50 categories

Good:

Clear and simple

3. Label Everything

Always include:

Title
Axis labels
Units

Practice Exercises

Create a sales line chart.
Create product comparison bar chart.
Analyze student marks.
Create histogram of ages.
Build a small dashboard.
Visualize a CSV dataset.

Chapter 43: Machine Learning with Python – Complete Introduction

What is Machine Learning?

Machine Learning (ML) is a branch of Artificial Intelligence (AI) that allows computers to learn from data and make decisions or predictions without being explicitly programmed.

Traditional programming:

Rules + Data

      |

      ↓

Program

      |

      ↓

Output

Machine Learning:

Data + Answers

      |

      ↓

Machine Learning Algorithm

      |

      ↓

Learned Model

      |

      ↓

Prediction

AI vs Machine Learning vs Deep Learning

Artificial Intelligence (AI)

AI is the broad field of making machines behave intelligently.

Examples:

Voice assistants
Self-driving cars
Recommendation systems

Machine Learning (ML)

ML allows systems to learn patterns from data.

Examples:

Spam detection
Price prediction
Customer analysis

Deep Learning (DL)

Deep Learning is a subset of ML using neural networks.

Examples:

Image recognition
ChatGPT-like systems
Speech recognition

Relationship:

Artificial Intelligence

        |

        |

Machine Learning

        |

        |

Deep Learning

Why Use Machine Learning?

Machine Learning is useful when:

Rules are difficult to write manually
Large amounts of data exist
Predictions are needed

Examples:

Email Spam Detection

Input:

Email content

Output:

Spam / Not Spam

House Price Prediction

Input:

Area
Location
Rooms
Age

Output:

House Price

Machine Learning Workflow

A typical ML project:

1. Collect Data

        |

2. Clean Data

        |

3. Explore Data

        |

4. Prepare Features

        |

5. Select Algorithm

        |

6. Train Model

        |

7. Evaluate Model

        |

8. Deploy Model

Types of Machine Learning

There are three main types:

Supervised Learning
Unsupervised Learning
Reinforcement Learning

1. Supervised Learning

The model learns from labeled data.

Meaning:

Input + Correct Output are provided.

Example:

Training data:

House Size	Price
1000 sq ft	50000
1500 sq ft	75000

The model learns:

Size → Price relationship

Types of Supervised Learning

A. Regression

Used for predicting numbers.

Examples:

House price
Temperature
Sales prediction

Output:

B. Classification

Used for categories.

Examples:

Spam / Not Spam
Disease / Healthy
Cat / Dog

Output:

Category

2. Unsupervised Learning

The model finds hidden patterns without labels.

Example:

Customer data:

Age
Income
Shopping habits

Model finds groups:

Group 1: Premium Customers

Group 2: Regular Customers

Group 3: New Customers

Common algorithms:

K-Means Clustering
PCA
Association Rules

3. Reinforcement Learning

A system learns by interacting with an environment.

Concept:

Agent

 |

Action

 |

Environment

 |

Reward/Punishment

 |

Learning

Examples:

Game AI
Robotics
Autonomous systems

Important Machine Learning Terms

Dataset

A collection of data used for learning.

Example:

students.csv

Features

Input variables used for prediction.

Example:

House prediction:

Features:

Area
Rooms
Location

Label / Target

The value we want to predict.

Example:

Price

Model

A mathematical system that learns patterns.

Example:

Input Data

↓

Model

↓

Prediction

Training Data

Data used to teach the model.

Example:

80% of dataset

Testing Data

Data used to check performance.

Example:

20% of dataset

Machine Learning Libraries in Python

Scikit-learn

TensorFlow

Used for:

Deep learning
Neural networks

Install:

pip install tensorflow

PyTorch

Used for:

Research
Deep learning

First Machine Learning Program

Problem:

Predict student marks based on study hours.

Data:

Hours	Marks
1	20
2	40
3	60
4	80

Import Libraries

from sklearn.linear_model import LinearRegression

Create Training Data

X=[
[1],
[2],
[3],
[4]
]


y=[
20,
40,
60,
80
]

Here:

X = Features (Hours)

y = Target (Marks)

Create Model

model=LinearRegression()

Train Model

model.fit(
X,
y
)

The model learns:

1 hour → 20 marks
2 hours → 40 marks

Make Prediction

Predict marks for 5 hours:

prediction=model.predict(
[[5]]
)


print(prediction)

Output:

[100]

Machine Learning Algorithms Overview

Regression Algorithms

Used for numerical prediction:

Linear Regression
Polynomial Regression
Ridge Regression
Lasso Regression

Classification Algorithms

Used for categories:

Logistic Regression
Decision Tree
Random Forest
Support Vector Machine
K-Nearest Neighbors

Clustering Algorithms

Used for grouping:

K-Means
DBSCAN
Hierarchical Clustering

Model Training Concept

Example:

Training Data

       |

       ↓

Algorithm

       |

       ↓

Learn Pattern

       |

       ↓

Model

Overfitting and Underfitting

Overfitting

Model memorizes training data.

Problem:

Training Accuracy: 99%

Testing Accuracy: 60%

Underfitting

Model is too simple.

Example:

Training Accuracy: 60%

Testing Accuracy: 60%

Machine Learning Project Example

Customer Churn Prediction

Goal:

Predict whether customers leave a company.

Data:

Age
Usage
Monthly Bill
Contract Type

Steps:

Load data
Clean data
Select features
Train model
Test accuracy
Deploy prediction API

Machine Learning Applications

Healthcare

Disease prediction
Medical image analysis

Finance

Fraud detection
Risk analysis

Marketing

Customer recommendations
Sales forecasting

Technology

Search engines
Voice assistants

Practice Exercises

Install Scikit-learn.
Create a linear regression model.
Predict house prices.
Build a classification model.
Split data into training/testing sets.
Measure model accuracy.

Chapter 44: Supervised Learning Algorithms with Python

What is Supervised Learning?

Supervised Learning is a machine learning method where the model learns from labeled data.

The dataset contains:

Input data (Features X)
Correct answer (Target y)

Example:

Hours Studied	Marks
1	20
2	40
3	60
4	80

The model learns:

Study Hours → Marks Prediction

Supervised Learning Workflow

Collect Data

     ↓

Split Dataset

     ↓

Train Model

     ↓

Test Model

     ↓

Evaluate Performance

     ↓

Make Predictions

Splitting Dataset

Usually:

Training Data → 80%

Testing Data → 20%

Training teaches the model.

Testing checks if the model works on new data.

Train-Test Split in Python

Install:

pip install scikit-learn

Example:

from sklearn.model_selection import train_test_split


X=[
[1],
[2],
[3],
[4],
[5]
]


y=[
10,
20,
30,
40,
50
]


X_train,X_test,y_train,y_test=train_test_split(
X,
y,
test_size=0.2
)

1. Linear Regression

What is Linear Regression?

Linear Regression predicts a continuous numerical value.

Examples:

House price prediction
Salary prediction
Sales forecasting

Formula:

y = mx + c

Where:

y = prediction
x = input
m = slope
c = intercept

Linear Regression Example

Problem:

Predict salary based on experience.

Data:

Experience	Salary
1	30000
2	40000
3	50000
4	60000

Import Library

from sklearn.linear_model import LinearRegression

Create Data

X=[
[1],
[2],
[3],
[4]
]


y=[
30000,
40000,
50000,
60000
]

Create Model

model=LinearRegression()

Train Model

model.fit(
X,
y
)

Prediction

result=model.predict(
[[5]]
)


print(result)

Output:

Checking Model Coefficients

Slope:

model.coef_

Intercept:

model.intercept_

Regression Evaluation

Mean Absolute Error (MAE)

Average prediction error.

from sklearn.metrics import mean_absolute_error

Example:

mean_absolute_error(
actual,
prediction
)

Mean Squared Error (MSE)

Punishes large errors.

from sklearn.metrics import mean_squared_error

R² Score

Measures model quality.

Range:

0 → Poor

1 → Perfect

Example:

from sklearn.metrics import r2_score

2. Logistic Regression

What is Logistic Regression?

Despite the name, it is a classification algorithm.

Used for:

Yes/No prediction
True/False prediction
Category prediction

Examples:

Spam detection
Disease prediction
Customer churn

Example

Predict if student passes.

Data:

Hours	Pass
1	No
2	No
5	Yes
6	Yes

Logistic Regression Code

from sklearn.linear_model import LogisticRegression


X=[
[1],
[2],
[5],
[6]
]


y=[
0,
0,
1,
1
]


model=LogisticRegression()


model.fit(
X,
y
)

Prediction:

model.predict(
[[4]]
)

Output:

Classification Metrics

Accuracy

Percentage of correct predictions.

Formula:

Correct Predictions / Total Predictions

Example:

from sklearn.metrics import accuracy_score


accuracy_score(
y_test,
prediction
)

Confusion Matrix

Shows:

Correct predictions
Wrong predictions

Example:

from sklearn.metrics import confusion_matrix


confusion_matrix(
y_test,
prediction
)

3. Decision Tree Algorithm

What is Decision Tree?

A decision tree makes decisions using rules.

Example:

Age > 18?

     |

   Yes ---- Buy Product

     |

    No ---- Do Not Buy

Decision Tree Types

Classification Tree

Output:

Regression Tree

Output:

Numbers

Example:

House Price

Decision Tree Example

from sklearn.tree import DecisionTreeClassifier


X=[
[20],
[25],
[30],
[35]
]


y=[
0,
1,
1,
1
]


model=DecisionTreeClassifier()


model.fit(
X,
y
)

Prediction:

model.predict(
[[22]]
)

Decision Tree Advantages

✅ Easy to understand
✅ Handles different data types
✅ Requires little preprocessing

Decision Tree Disadvantages

❌ Can overfit
❌ Sensitive to data changes

4. Random Forest Algorithm

What is Random Forest?

Random Forest combines many decision trees.

Concept:

Tree 1
   \
Tree 2 ---- Prediction
   /
Tree 3

Multiple trees vote for the final answer.

Random Forest Example

from sklearn.ensemble import RandomForestClassifier


model=RandomForestClassifier()


model.fit(
X_train,
y_train
)

Random Forest Advantages

✅ High accuracy
✅ Reduces overfitting
✅ Works with large datasets

Applications

Fraud detection
Medical diagnosis
Customer prediction

5. K-Nearest Neighbors (KNN)

What is KNN?

KNN predicts based on similar nearby data points.

Example:

Nearby customers

      ↓

Same category

KNN Example

from sklearn.neighbors import KNeighborsClassifier


model=KNeighborsClassifier(
n_neighbors=3
)


model.fit(
X_train,
y_train
)

KNN Advantages

✅ Simple algorithm
✅ Easy to implement

KNN Disadvantages

❌ Slow with large data
❌ Sensitive to scaling

6. Support Vector Machine (SVM)

What is SVM?

SVM creates the best boundary between classes.

Example:

Class A  |  Class B

---------|---------

Best separating line

SVM Example

from sklearn.svm import SVC


model=SVC()


model.fit(
X_train,
y_train
)

SVM Applications

Image classification
Text classification
Pattern recognition

Feature Scaling

Some algorithms need data scaling.

Example:

Before:

Age = 25

Salary = 500000

Salary dominates.

After scaling:

Age = 0.2

Salary = 0.8

StandardScaler

Example:

from sklearn.preprocessing import StandardScaler


scaler=StandardScaler()


X_scaled=scaler.fit_transform(
X
)

Comparing Supervised Algorithms

Algorithm	Type	Used For
Linear Regression	Regression	Price prediction
Logistic Regression	Classification	Yes/No prediction
Decision Tree	Both	Rule-based decisions
Random Forest	Both	High accuracy models
KNN	Classification	Similarity problems
SVM	Classification	Complex boundaries

Real-World Projects

1. House Price Prediction

Algorithm:

Linear Regression
Random Forest

Features:

Area
Rooms
Location

2. Spam Email Detection

Algorithm:

Logistic Regression
SVM

Features:

Words
Email patterns

3. Customer Churn Prediction

Algorithm:

Random Forest
Decision Tree

Features:

Usage
Payment history

Practice Exercises

Build salary prediction using Linear Regression.
Create spam classifier.
Train Decision Tree model.
Compare Random Forest and KNN.
Calculate accuracy score.
Visualize predictions.

Chapter 46: Feature Engineering and Data Preprocessing

What is Data Preprocessing?

Data preprocessing is the process of preparing raw data before giving it to a machine learning model.

Real-world data is usually:

Incomplete
Incorrect
Unstructured
Contains missing values
Contains different formats

Machine learning models work better with clean and prepared data.

Machine Learning Data Pipeline

Raw Data

   ↓

Data Cleaning

   ↓

Data Transformation

   ↓

Feature Engineering

   ↓

Feature Selection

   ↓

Machine Learning Model

What is Feature?

A feature is an input variable used by a model to make predictions.

Example:

House price prediction:

Feature	Value
Area	2000 sq ft
Rooms	3
Location	Delhi

Target:

Price

What is Feature Engineering?

Feature engineering means creating better input features from existing data.

Example:

Original data:

Date:
20-07-2026

Create new features:

Day = 20

Month = 7

Year = 2026

Why Feature Engineering is Important?

Good features improve:

✅ Accuracy
✅ Model performance
✅ Learning speed

Poor features can cause:

❌ Wrong predictions
❌ Low accuracy

1. Data Cleaning

Data cleaning removes errors from datasets.

Common problems:

Missing values
Duplicate records
Wrong formats
Outliers

Loading Dataset

Example:

import pandas as pd


df=pd.read_csv(
"data.csv"
)

Checking Data

View first rows:

df.head()

Information:

df.info()

Statistics:

df.describe()

2. Handling Missing Values

Example dataset:

Name	Age	Salary
John	25	50000
Alex		60000
Sam	30

Missing values:

Age = empty

Salary = empty

Detect Missing Values

df.isnull()

Count:

df.isnull().sum()

Removing Missing Values

Remove rows:

df.dropna()

Before:

1000 rows

After:

950 rows

Filling Missing Values

Fill with Mean

Example:

df["Age"].fillna(
df["Age"].mean()
)

Fill with Median

df["Age"].fillna(
df["Age"].median()
)

Fill with Mode

Used for categories.

Example:

df["City"].fillna(
df["City"].mode()[0]
)

3. Handling Duplicate Data

Check duplicates:

df.duplicated()

Remove:

df.drop_duplicates()

4. Handling Incorrect Data

Example:

Wrong:

Age = -5

Correct:

Age = Positive number

Find:

df[df["Age"]<0]

5. Handling Outliers

What are Outliers?

Outliers are unusual values.

Example:

Normal salaries:

30000
40000
50000

Outlier:

Detecting Outliers

Using IQR method:

Q1 = 25%

Q3 = 75%

IQR = Q3 - Q1

Python:

Q1=df["Salary"].quantile(
0.25
)


Q3=df["Salary"].quantile(
0.75
)

6. Encoding Categorical Data

Machine learning models understand numbers, not text.

Example:

Before:

City
Delhi
Mumbai
Kolkata

After:

City
0
1
2

Label Encoding

Converts categories into numbers.

Example:

from sklearn.preprocessing import LabelEncoder


encoder=LabelEncoder()


df["City"]=encoder.fit_transform(
df["City"]
)

One-Hot Encoding

Creates separate columns.

Before:

City

Delhi
Mumbai

After:

Delhi	Mumbai
1	0
0	1

Python:

pd.get_dummies(
df["City"]
)

7. Feature Scaling

Why Scaling?

Different features may have different ranges.

Example:

Age:

20-60


Salary:

20000-500000

Salary dominates.

Scaling puts values into similar ranges.

Types of Scaling

1. Standardization

Transforms data:

Mean = 0

Standard deviation = 1

Formula:

z = (x - mean) / standard deviation

Python:

from sklearn.preprocessing import StandardScaler


scaler=StandardScaler()


X_scaled=scaler.fit_transform(
X
)

2. Normalization

Converts values:

0 to 1 range

Python:

from sklearn.preprocessing import MinMaxScaler


scaler=MinMaxScaler()


X_scaled=scaler.fit_transform(
X
)

8. Feature Selection

What is Feature Selection?

Choosing the most important features.

Example:

Dataset:

100 features

Select:

10 useful features

Why Feature Selection?

Benefits:

✅ Faster training
✅ Less complexity
✅ Better accuracy

Methods of Feature Selection

1. Correlation

Find relationship between features.

Example:

df.corr()

2. Feature Importance

Used with tree models.

Example:

model.feature_importances_

3. SelectKBest

Select top features.

Example:

from sklearn.feature_selection import SelectKBest

9. Creating New Features

Example:

Dataset:

Date

Create:

Day
Month
Year

Example:

df["Year"]=pd.to_datetime(
df["Date"]
).dt.year

10. Data Transformation

Making data suitable for models.

Examples:

Scaling
Encoding
Log transformation

Log Transformation

Used for highly uneven data.

Example:

Before:

After:

Complete Preprocessing Example

Dataset:

customer.csv

Columns:

Age
Salary
City
Purchased

Step 1: Load Data

df=pd.read_csv(
"customer.csv"
)

Step 2: Fill Missing Values

df["Age"].fillna(
df["Age"].mean(),
inplace=True
)

Step 3: Encode City

df=pd.get_dummies(
df,
columns=["City"]
)

Step 4: Scale Data

scaler=StandardScaler()

X=scaler.fit_transform(
X
)

Machine Learning Pipeline

Scikit-learn provides pipelines.

Example:

from sklearn.pipeline import Pipeline


pipeline=Pipeline([

("scaler",StandardScaler()),

("model",LinearRegression())

])

Real-World Preprocessing Examples

House Price Prediction

Cleaning:

Missing prices
Encode locations
Scale area

Customer Prediction

Cleaning:

Missing age
Convert categories
Select important features

Medical Data

Cleaning:

Remove errors
Normalize measurements
Handle missing tests

Practice Exercises

Load a CSV dataset.
Find missing values.
Remove duplicates.
Encode categorical columns.
Apply feature scaling.
Select important features.
Create a preprocessing pipeline.

Chapter 47: Model Evaluation and Optimization

What is Model Evaluation?

Model evaluation is the process of measuring how well a machine learning model performs.

After training a model, we need to answer:

Is the model accurate?
Does it work on new data?
Is it overfitting?
Can we improve it?

Machine Learning Model Workflow

Dataset

   ↓

Preprocessing

   ↓

Train Model

   ↓

Make Predictions

   ↓

Evaluate Performance

   ↓

Optimize Model

Why Model Evaluation is Important?

A model can perform well on training data but fail on new data.

Example:

Training:

Accuracy = 99%

Testing:

Accuracy = 60%

Problem:

Overfitting

Training vs Testing Performance

Good Model

Training Accuracy: 90%

Testing Accuracy: 88%

The model generalizes well.

Overfitting

Training Accuracy: 99%

Testing Accuracy: 70%

The model memorized training data.

Underfitting

Training Accuracy: 60%

Testing Accuracy: 55%

The model is too simple.

Classification Evaluation Metrics

Used when output is categories.

Examples:

Spam / Not Spam
Disease / Healthy
Cat / Dog

1. Accuracy

Accuracy measures how many predictions are correct.

Formula:

Accuracy =
Correct Predictions / Total Predictions

Example:

100 predictions

90 correct

Accuracy = 90%

Python:

from sklearn.metrics import accuracy_score


accuracy_score(
y_test,
prediction
)

Problem with Accuracy

Accuracy can be misleading.

Example:

Disease detection:

1000 people

950 healthy

50 sick

A model predicting everyone healthy:

Accuracy = 95%

But it misses all sick people.

2. Confusion Matrix

A confusion matrix shows prediction results.

Example:

                Predicted

             Yes       No

Actual Yes    TP        FN

Actual No     FP        TN

Meaning:

True Positive (TP)

Correctly predicted positive.

Example:

Sick person detected

True Negative (TN)

Correctly predicted negative.

Example:

Healthy person detected

False Positive (FP)

Wrong positive prediction.

Example:

Healthy person marked sick

False Negative (FN)

Wrong negative prediction.

Example:

Sick person missed

Python:

from sklearn.metrics import confusion_matrix


confusion_matrix(
y_test,
prediction
)

3. Precision

Precision answers:

Of all predicted positives, how many were actually positive?

Formula:

Precision =
TP / (TP + FP)

Example:

Spam detection:

100 emails marked spam

90 actually spam

Precision:

90%

Python:

from sklearn.metrics import precision_score


precision_score(
y_test,
prediction
)

4. Recall

Recall answers:

Of all actual positives, how many did the model find?

Formula:

Recall =
TP / (TP + FN)

Important for:

Disease detection
Fraud detection

Python:

from sklearn.metrics import recall_score


recall_score(
y_test,
prediction
)

5. F1 Score

F1 combines:

Precision
Recall

Formula:

F1 =
2 × (Precision × Recall)
/ 
(Precision + Recall)

Python:

from sklearn.metrics import f1_score


f1_score(
y_test,
prediction
)

Classification Report

Shows all metrics together.

from sklearn.metrics import classification_report


print(
classification_report(
y_test,
prediction
)
)

Output:

precision
recall
f1-score
accuracy

Regression Evaluation Metrics

Used when output is a number.

Examples:

Price prediction
Sales prediction
Temperature prediction

1. Mean Absolute Error (MAE)

Average difference between actual and predicted values.

Example:

Actual:
100

Prediction:
95

Error:
5

Python:

from sklearn.metrics import mean_absolute_error


mean_absolute_error(
y_test,
prediction
)

2. Mean Squared Error (MSE)

Squares errors.

Large errors are punished more.

from sklearn.metrics import mean_squared_error


mean_squared_error(
y_test,
prediction
)

3. Root Mean Squared Error (RMSE)

Square root of MSE.

Formula:

RMSE = √MSE

Python:

import numpy as np


rmse=np.sqrt(
mean_squared_error(
y_test,
prediction
)
)

4. R² Score

Shows how well the model explains data.

Range:

0 = Poor

1 = Perfect

Python:

from sklearn.metrics import r2_score


r2_score(
y_test,
prediction
)

Cross Validation

What is Cross Validation?

Cross validation tests a model multiple times using different data splits.

Instead of:

Train → Test once

It does:

Train/Test

Train/Test

Train/Test

Train/Test

K-Fold Cross Validation

Example:

5-Fold:

Dataset

Fold 1
Fold 2
Fold 3
Fold 4
Fold 5

Each fold becomes testing data once.

Python:

from sklearn.model_selection import cross_val_score


scores=cross_val_score(
model,
X,
y,
cv=5
)


print(scores)

Benefits of Cross Validation

✅ Better accuracy estimate
✅ Uses all data
✅ Reduces random errors

Hyperparameter Tuning

What are Hyperparameters?

Settings chosen before training.

Examples:

Random Forest:

Number of trees

Tree depth

KNN:

Number of neighbors

Grid Search

Grid Search tests many combinations.

Example:

from sklearn.model_selection import GridSearchCV

Example:

parameters={

"n_neighbors":[3,5,7],

}

Create search:

grid=GridSearchCV(

model,

parameters,

cv=5

)

Train:

grid.fit(
X_train,
y_train
)

Best Parameters:

grid.best_params_

Random Search

Randomly tries combinations.

Useful when:

Many parameters exist
Dataset is large

Python:

from sklearn.model_selection import RandomizedSearchCV

Regularization

Used to reduce overfitting.

Main methods:

L1 Regularization

Also called:

Lasso

Removes unnecessary features.

L2 Regularization

Also called:

Ridge

Reduces large weights.

Ensemble Learning

Combines multiple models.

Example:

Model 1

Model 2

Model 3

   ↓

Final Prediction

Examples:

Random Forest
Gradient Boosting
XGBoost

Gradient Boosting

Builds models step-by-step.

Popular algorithms:

Gradient Boosting
XGBoost
LightGBM

Used in:

Competitions
Finance
Business prediction

Model Optimization Workflow

Train Model

      ↓

Evaluate

      ↓

Find Problems

      ↓

Tune Parameters

      ↓

Retrain

      ↓

Compare Results

Complete Model Evaluation Example

from sklearn.metrics import accuracy_score
from sklearn.metrics import classification_report


model.fit(
X_train,
y_train
)


prediction=model.predict(
X_test
)


print(
accuracy_score(
y_test,
prediction
)
)


print(
classification_report(
y_test,
prediction
)
)

Real-World Optimization Examples

Fraud Detection

Focus:

High Recall
Reduce missed fraud

Medical Diagnosis

Focus:

Recall
F1 Score

Recommendation System

Focus:

Precision
User satisfaction

Practice Exercises

Calculate accuracy of a classifier.
Create confusion matrix.
Compare precision and recall.
Evaluate regression model using MAE and RMSE.
Apply cross-validation.
Tune model using GridSearchCV.
Reduce overfitting.

Chapter 48: Deep Learning with Python

What is Deep Learning?

Deep Learning is a subset of Machine Learning that uses artificial neural networks to learn from large amounts of data.

Deep Learning is inspired by the human brain.

Example:

Human Brain

Neurons

   ↓

Artificial Neural Network

   ↓

Machine Learning Model

AI → ML → Deep Learning Relationship

Artificial Intelligence

        |

        |

Machine Learning

        |

        |

Deep Learning

        |

        |

Neural Networks

Why Deep Learning?

Deep Learning is useful when data is:

Very large
Complex
Unstructured

Examples:

Images
Videos
Audio
Text

Applications of Deep Learning

Image Recognition

Examples:

Face recognition
Medical image analysis
Object detection

Natural Language Processing

Examples:

Chatbots
Translation
Voice assistants

Autonomous Vehicles

Examples:

Road detection
Object recognition

Recommendation Systems

Examples:

Movies
Products
Music

Traditional Machine Learning vs Deep Learning

Machine Learning	Deep Learning
Small data works	Needs large data
Manual feature selection	Learns features automatically
Simple models	Complex neural networks
Less computing power	Requires GPUs

What is an Artificial Neural Network (ANN)?

An ANN is a computing system inspired by biological neurons.

A neural network contains:

Input Layer
Hidden Layers
Output Layer

Structure:

Input Layer

     ↓

Hidden Layer

     ↓

Hidden Layer

     ↓

Output Layer

Artificial Neuron

A neuron receives inputs and produces output.

Example:

Input 1
   \
    \
Input 2 ---> Neuron ---> Output
    /
Input 3

Neural Network Layers

1. Input Layer

Receives data.

Example:

Image:

Pixels

2. Hidden Layers

Learn patterns.

Example:

First layer:

Edges

Second layer:

Shapes

Third layer:

Objects

3. Output Layer

Produces final result.

Example:

Cat = 90%

Dog = 10%

Neural Network Learning Process

Input Data

     ↓

Forward Propagation

     ↓

Prediction

     ↓

Calculate Error

     ↓

Backpropagation

     ↓

Update Weights

     ↓

Better Prediction

Important Neural Network Terms

Weights

Numbers that control importance of inputs.

Example:

Feature A → Weight 0.8

Feature B → Weight 0.2

Bias

Extra value added to improve learning.

Activation Function

Decides whether a neuron should activate.

Common Activation Functions

1. ReLU

Most common.

Formula:

max(0,x)

Used in hidden layers.

2. Sigmoid

Output:

0 to 1

Used for binary classification.

Example:

Spam probability = 0.95

3. Softmax

Used for multiple classes.

Example:

Cat: 0.8

Dog: 0.15

Bird: 0.05

Deep Learning Libraries in Python

TensorFlow

Developed by Google.

Used for:

Neural networks
Production AI systems

Install:

pip install tensorflow

Keras

High-level API inside TensorFlow.

Used for:

Fast model creation
Beginners

PyTorch

Developed by Meta.

Used for:

Research
Advanced AI

Install:

pip install torch

Building First Neural Network Using Keras

Import Libraries

import tensorflow as tf

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense

Create Model

model=Sequential()

Add Layers

Input + Hidden Layer:

model.add(
Dense(
10,
activation="relu",
input_shape=(5,)
)
)

Output Layer:

model.add(
Dense(
1,
activation="sigmoid"
)
)

Compile Model

Before training:

model.compile(

optimizer="adam",

loss="binary_crossentropy",

metrics=["accuracy"]

)

Train Model

model.fit(

X_train,

y_train,

epochs=10

)

Make Predictions

prediction=model.predict(
X_test
)

Understanding Training

Epoch

One complete pass through the dataset.

Example:

Epoch 1

↓

Epoch 2

↓

Epoch 3

Batch Size

Number of samples processed together.

Example:

Dataset = 1000 images

Batch size = 32

Loss Function

Measures model error.

Goal:

Reduce Loss

Examples:

Classification:

Binary Cross Entropy

Regression:

Mean Squared Error

Optimizers

Optimizers update model weights.

Popular:

Gradient Descent

Basic optimizer.

Adam

Most commonly used.

Example:

optimizer="adam"

Types of Deep Learning Networks

1. Feed Forward Neural Network

Basic neural network.

Used for:

Classification
Regression

2. Convolutional Neural Network (CNN)

Used for images.

Applications:

Face recognition
Object detection
Medical imaging

Structure:

Image

 ↓

Convolution

 ↓

Features

 ↓

Prediction

3. Recurrent Neural Network (RNN)

Used for sequences.

Examples:

Text
Speech
Time series

4. LSTM

Improved RNN.

Used for:

Language models
Stock prediction
Long text processing

5. Transformers

Modern AI architecture.

Used in:

ChatGPT
Translation systems
Large Language Models

Image Classification Example

Problem:

Classify:

Cat or Dog

Input:

Image pixels

CNN learns:

Edges

↓

Shapes

↓

Animal features

↓

Prediction

Deep Learning Project Workflow

Collect Data

      ↓

Prepare Dataset

      ↓

Build Neural Network

      ↓

Train Model

      ↓

Evaluate

      ↓

Deploy

Overfitting in Deep Learning

Deep networks can memorize data.

Solutions:

Dropout

Randomly disables neurons during training.

Example:

Dropout(0.5)

Data Augmentation

Creates more training examples.

Example:

Image:

Rotate

Flip

Crop

Early Stopping

Stops training when performance stops improving.

GPU and Deep Learning

Deep learning requires high computation.

CPU:

General calculations

GPU:

Parallel matrix calculations

Popular:

NVIDIA GPUs
CUDA

Real-World Deep Learning Projects

1. Image Classifier

Technology:

CNN
TensorFlow

2. Chatbot

Technology:

NLP
Transformers

3. Face Recognition

Technology:

CNN
Computer Vision

4. Recommendation Engine

Technology:

Neural Networks

Practice Exercises

Install TensorFlow.
Create a simple neural network.
Train a binary classifier.
Experiment with activation functions.
Add dropout layers.
Create an image classification model.

Chapter 49: Neural Networks and AI Basics

What is an Artificial Neural Network?

An Artificial Neural Network (ANN) is a machine learning model inspired by the structure of the human brain.

It consists of connected artificial neurons that learn patterns from data.

Example:

Input Data

    ↓

Neural Network

    ↓

Learn Patterns

    ↓

Prediction

Biological Neuron vs Artificial Neuron

Human Brain Neuron

A biological neuron has:

Dendrites → Receive signals
Cell body → Process signals
Axon → Send signals

Artificial Neuron

A computer neuron has:

Inputs
Weights
Bias
Activation function
Output

Structure:

Input

 x1 ─── Weight

 x2 ─── Weight  ───> Neuron ───> Output

 x3 ─── Weight

Mathematical Model of a Neuron

A neuron calculates:

Output = Activation(Weights × Inputs + Bias)

Formula:

y = f(w1x1 + w2x2 + w3x3 + b)

Where:

x = input values
w = weights
b = bias
f = activation function
y = output

Understanding Weights

Weights decide the importance of input features.

Example:

Predict house price:

Area       Weight = 0.8

Location   Weight = 0.6

Age        Weight = -0.3

Meaning:

Larger area increases price
Older house may reduce price

Understanding Bias

Bias helps the model adjust output.

Without bias:

Output = Weight × Input

With bias:

Output = Weight × Input + Bias

Bias improves flexibility.

Neural Network Architecture

A neural network contains:

Input Layer
Hidden Layers
Output Layer

Example:

Input Layer

[ x1 ]
[ x2 ]
[ x3 ]

     ↓

Hidden Layer

[ ● ]
[ ● ]
[ ● ]

     ↓

Output Layer

[ y ]

Forward Propagation

Forward propagation means sending input data through the network to generate a prediction.

Steps:

Input Data

    ↓

Calculate Weighted Sum

    ↓

Apply Activation Function

    ↓

Generate Output

Example Forward Propagation

Input:

x = 5

Weight:

w = 2

Bias:

b = 1

Calculation:

z = wx + b

z = (2 × 5) + 1

z = 11

Activation:

Output = activation(11)

Activation Functions

Activation functions allow neural networks to learn complex patterns.

1. ReLU Activation

Formula:

f(x)=max(0,x)

Example:

Input = -5

Output = 0

Input = 10

Output = 10

Used in:

Hidden layers
Deep networks

2. Sigmoid Activation

Output range:

0 to 1

Formula:

1/(1+e^-x)

Used for:

Binary classification

Example:

Email Spam Probability

0.95 = Spam

3. Tanh Activation

Range:

-1 to 1

Used in:

Some neural networks
RNN models

4. Softmax Activation

Used for multiple classes.

Example:

Image classification:

Cat     0.80

Dog     0.15

Bird    0.05

Total:

1.0

Loss Function

What is Loss?

Loss measures how wrong the model prediction is.

Example:

Actual:

Prediction:

Error:

The model tries to reduce this error.

Common Loss Functions

Mean Squared Error (MSE)

Used for regression.

Formula:

(actual - prediction)²

Example:

Price prediction
Sales prediction

Binary Cross Entropy

Used for:

Yes/No classification

Example:

Spam / Not Spam

Categorical Cross Entropy

Used for:

Multiple classes

Example:

Cat
Dog
Bird

Backpropagation

What is Backpropagation?

Backpropagation is the process of improving the neural network by updating weights after calculating errors.

Flow:

Prediction

    ↓

Calculate Error

    ↓

Send Error Back

    ↓

Update Weights

    ↓

Better Prediction

How Backpropagation Works

Steps:

Step 1

Network makes prediction.

Step 2

Calculate loss.

Step 3

Find which weights caused the error.

Step 4

Adjust weights.

Step 5

Repeat many times.

Gradient Descent

What is Gradient Descent?

Gradient descent is an optimization algorithm used to reduce loss.

Goal:

Find minimum error

Visual:

Loss

 |
 |
 |        *
 |
 |     *
 |
 |  *
 |
 |_____________

     Minimum

Learning Rate

Learning rate controls how much weights change.

Example:

Small learning rate:

Slow learning

Large learning rate:

May miss best solution

Example:

learning_rate=0.001

Epochs

An epoch is one complete pass through training data.

Example:

Dataset:

1000 images

One epoch:

Model sees all 1000 images once

Training:

Epoch 1
Epoch 2
Epoch 3
...

Batch Size

Number of samples processed before updating weights.

Example:

Dataset:

10000 samples

Batch size:

The model updates after every 100 samples.

Optimizers

Optimizers control weight updates.

1. SGD (Stochastic Gradient Descent)

Basic optimizer.

optimizer="sgd"

2. Adam Optimizer

Neural Network Example Using Keras

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense


model=Sequential()


model.add(
Dense(
8,
activation="relu",
input_shape=(4,)
)
)


model.add(
Dense(
1,
activation="sigmoid"
)
)


model.compile(
optimizer="adam",
loss="binary_crossentropy",
metrics=["accuracy"]
)

Improving Neural Networks

1. More Data

More examples help learning.

2. Better Features

Good input improves results.

3. More Layers

Creates deeper networks.

4. Regularization

Prevents overfitting.

Methods:

Dropout
L1/L2 regularization

5. Batch Normalization

Improves training stability.

Example:

BatchNormalization()

Neural Network Types

Feed Forward Neural Network

Basic network.

Used for:

Classification
Regression

CNN (Convolutional Neural Network)

Used for:

Images
Video

RNN (Recurrent Neural Network)

Used for:

Text
Sequence data

LSTM

Improved RNN.

Used for:

Language
Time series

Transformer Networks

Modern AI architecture.

Used in:

ChatGPT
Translation
Large Language Models

Real AI Applications

Healthcare

Disease prediction
Medical imaging

Finance

Fraud detection
Market analysis

Transportation

Autonomous vehicles
Traffic prediction

Entertainment

Recommendations
Content generation

Practice Exercises

Understand neuron calculations manually.
Build a simple ANN.
Experiment with activation functions.
Change learning rates.
Compare optimizers.
Train a classification neural network.

Chapter 50: Natural Language Processing (NLP) with Python

What is Natural Language Processing (NLP)?

Natural Language Processing (NLP) is a branch of Artificial Intelligence that enables computers to understand, process, and generate human language.

Examples:

Chatbots
Voice assistants
Translation systems
Sentiment analysis
Text summarization

Human Language vs Computer Language

Humans understand:

"I love this movie"

Computers understand numbers:

[0.25, 0.78, 0.41]

NLP converts text into numerical representations.

Applications of NLP

1. Chatbots

Examples:

Customer support bots
AI assistants

2. Sentiment Analysis

Determines emotion from text.

Example:

Input:

"The product is amazing"

Output:

Positive

3. Language Translation

Example:

English

↓

French

4. Speech Recognition

Example:

Voice

↓

Text

5. Text Classification

Examples:

Spam detection
News categorization
Topic detection

NLP Pipeline

A typical NLP system:

Raw Text

   ↓

Text Cleaning

   ↓

Tokenization

   ↓

Feature Extraction

   ↓

Machine Learning Model

   ↓

Prediction

Step 1: Text Collection

Data examples:

Reviews
Tweets
Articles
Messages

Example:

text="Python is a great programming language"

Step 2: Text Cleaning

Raw text contains unnecessary information.

Common cleaning tasks:

Lowercase conversion
Removing punctuation
Removing numbers
Removing stop words

Convert to Lowercase

Before:

Python IS Great

After:

python is great

Python:

text=text.lower()

print(text)

Removing Punctuation

Example:

Before:

Hello!!!

After:

Hello

Python:

import string


text=text.translate(
str.maketrans(
"",
"",
string.punctuation
)
)

Removing Numbers

Example:

Before:

Python 2026

After:

Python

Step 3: Tokenization

What is Tokenization?

Breaking text into smaller parts called tokens.

Word Tokenization

Example:

Sentence:

Python is easy to learn

Tokens:

[
Python,
is,
easy,
to,
learn
]

Python using NLTK:

Install:

pip install nltk

Code:

import nltk

from nltk.tokenize import word_tokenize


text="Python is easy"


tokens=word_tokenize(
text
)

print(tokens)

Sentence Tokenization

Example:

Input:

Python is powerful.
It is popular.

Output:

[
"Python is powerful",

"It is popular"
]

Step 4: Stop Words Removal

What are Stop Words?

Common words that add little meaning.

Examples:

the
is
a
an
and

Example:

Before:

Python is a very powerful language

After:

Python powerful language

Python:

from nltk.corpus import stopwords


stop_words=set(
stopwords.words(
"english"
)
)

Step 5: Stemming

What is Stemming?

Converts words into their root form.

Examples:

playing → play

played → play

plays → play

Python:

from nltk.stem import PorterStemmer


stemmer=PorterStemmer()


stemmer.stem(
"playing"
)

Output:

play

Step 6: Lemmatization

Similar to stemming but more accurate.

Example:

better → good

running → run

Python:

from nltk.stem import WordNetLemmatizer


lemmatizer=WordNetLemmatizer()


lemmatizer.lemmatize(
"running"
)

Feature Extraction in NLP

Machine learning models cannot understand text directly.

Text must be converted into numbers.

1. Bag of Words (BoW)

Represents text by word frequency.

Example:

Sentence:

I love Python

Vocabulary:

I
love
Python

Vector:

[1,1,1]

Python:

from sklearn.feature_extraction.text import CountVectorizer


vectorizer=CountVectorizer()


X=vectorizer.fit_transform(
texts
)

2. TF-IDF

Term Frequency-Inverse Document Frequency

Measures word importance.

Common words:

the
is
a

Less important.

Unique words:

machine learning

More important.

Python:

from sklearn.feature_extraction.text import TfidfVectorizer


vectorizer=TfidfVectorizer()


X=vectorizer.fit_transform(
texts
)

3. Word Embeddings

Word embeddings represent words as vectors with meaning.

Example:

King

Queen

Man

Woman

The model understands relationships.

Popular methods:

Word2Vec
GloVe
FastText

NLP Machine Learning Models

Naive Bayes

Used for:

Spam detection
Text classification

Example:

from sklearn.naive_bayes import MultinomialNB

Logistic Regression

Used for:

Sentiment analysis

Support Vector Machine

Used for:

Text classification

Sentiment Analysis Project

Goal:

Classify reviews.

Dataset:

Review

Rating

Example:

Input:

"This product is excellent"

Output:

Positive

Steps:

Collect Reviews

      ↓

Clean Text

      ↓

Tokenize

      ↓

Convert Text to Numbers

      ↓

Train Model

      ↓

Predict Sentiment

NLP Libraries in Python

NLTK

Used for:

Tokenization
Stemming
Text processing

spaCy

Used for:

Industrial NLP
Named Entity Recognition

Transformers

Used for:

Modern AI models
Large Language Models

Named Entity Recognition (NER)

NER finds important information from text.

Example:

Sentence:

Apple was founded by Steve Jobs in California

Output:

Apple → Organization

Steve Jobs → Person

California → Location

Chatbot Basics

A chatbot workflow:

User Message

      ↓

NLP Processing

      ↓

Intent Detection

      ↓

Response Generation

      ↓

Answer

Modern NLP and Transformers

Transformers changed NLP.

Used in:

ChatGPT
Translation
Text generation

Important concepts:

Attention mechanism
Large Language Models (LLMs)
Pre-training
Fine-tuning

Real-World NLP Projects

Beginner Projects

Spam detector
Sentiment analyzer
Text classifier

Intermediate Projects

Chatbot
Resume analyzer
News classifier

Advanced Projects

Language translator
AI writing assistant
Question answering system

Practice Exercises

Clean a text dataset.
Perform tokenization.
Remove stop words.
Apply stemming.
Create TF-IDF features.
Build sentiment analysis model.
Create a simple chatbot.

Chapter 51: Computer Vision with Python

What is Computer Vision?

Computer Vision (CV) is a branch of Artificial Intelligence that allows computers to understand, analyze, and interpret images and videos.

Humans use eyes and brains to see and understand objects.

Computers use:

Cameras
Images
Algorithms
Deep Learning models

to understand visual information.

How Computer Vision Works

Image / Video

      ↓

Image Processing

      ↓

Feature Detection

      ↓

AI Model

      ↓

Prediction

Applications of Computer Vision

1. Face Recognition

Used in:

Phone unlocking
Security systems

2. Object Detection

Detect objects:

Examples:

Cars
People
Animals

3. Medical Imaging

Used for:

X-ray analysis
Disease detection

4. Self-Driving Cars

Computer vision detects:

Roads
Traffic signs
Vehicles

5. Augmented Reality

Examples:

Filters
Virtual objects

Image Basics

A digital image is a collection of pixels.

Example:

Image

↓

Pixels

↓

Numbers

Pixel

A pixel is the smallest unit of an image.

Example:

Small image:

10 × 10 pixels

= 100 pixels

Image Representation

Computers represent images as arrays.

Example:

[
[255,0,0],
[0,255,0],
[0,0,255]
]

Types of Images

1. Grayscale Image

Contains only brightness values.

Range:

0 → Black

255 → White

Example:

Black and White image

2. Color Image (RGB)

Contains three channels:

R = Red

G = Green

B = Blue

Example:

(255,0,0)

Red color

Computer Vision Libraries in Python

OpenCV

Pillow (PIL)

Used for:

Basic image operations

Install:

pip install pillow

TensorFlow / PyTorch

Used for:

Deep learning vision models

OpenCV Introduction

Import:

import cv2

Reading an Image

image=cv2.imread(
"photo.jpg"
)

Display Image

cv2.imshow(
"Image",
image
)


cv2.waitKey(0)

Save Image

cv2.imwrite(
"new_photo.jpg",
image
)

Image Properties

Check image size:

image.shape

Output:

(height,width,channels)

Example:

(1080,1920,3)

Reading Image Modes

Color Image

cv2.imread(
"image.jpg",
cv2.IMREAD_COLOR
)

Grayscale Image

cv2.imread(
"image.jpg",
cv2.IMREAD_GRAYSCALE
)

Image Processing Operations

1. Resize Image

Why resize?

Faster processing
Standard input size

Example:

resized=cv2.resize(
image,
(500,500)
)

2. Crop Image

Select part of image.

Example:

crop=image[
100:300,
100:300
]

3. Rotate Image

Example:

rotated=cv2.rotate(
image,
cv2.ROTATE_90_CLOCKWISE
)

4. Convert Color

RGB to Grayscale:

gray=cv2.cvtColor(
image,
cv2.COLOR_BGR2GRAY
)

5. Blur Image

Used to remove noise.

Example:

blur=cv2.GaussianBlur(
image,
(5,5),
0
)

Edge Detection

Edges show boundaries of objects.

Example:

Object

  ↓

Edges

  ↓

Shape detection

Canny Edge Detection

edges=cv2.Canny(
image,
100,
200
)

Image Thresholding

Converts image into black and white.

Example:

threshold=cv2.threshold(
gray,
127,
255,
cv2.THRESH_BINARY
)

Object Detection

Object detection finds:

What object exists?
Where it is located?

Output:

Car

Location:

(x,y,width,height)

Traditional Object Detection

Methods:

Haar Cascade
HOG

Face Detection Using OpenCV

Install:

OpenCV already includes classifiers.

Example:

face_detector=cv2.CascadeClassifier(
"haarcascade_frontalface_default.xml"
)

Detect faces:

faces=face_detector.detectMultiScale(
gray
)

Drawing Rectangle Around Face

for x,y,w,h in faces:

    cv2.rectangle(
    image,
    (x,y),
    (x+w,y+h),
    (255,0,0),
    2
    )

Image Classification

What is Image Classification?

Assigning a category to an image.

Example:

Input:

Image

Output:

Dog

CNN for Computer Vision

Convolutional Neural Networks are the most important deep learning models for images.

Structure:

Image

 ↓

Convolution Layer

 ↓

Pooling Layer

 ↓

Fully Connected Layer

 ↓

Prediction

Convolution Layer

Finds features:

First layers:

Edges

Middle layers:

Shapes

Deep layers:

Objects

Pooling Layer

Reduces image size.

Benefits:

Faster processing
Removes unnecessary details

Common:

Max Pooling
Average Pooling

Building CNN with TensorFlow

Import:

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Conv2D,MaxPooling2D,Dense,Flatten

Create Model:

model=Sequential()

Add Convolution:

model.add(
Conv2D(
32,
(3,3),
activation="relu",
input_shape=(64,64,3)
)
)

Pooling:

model.add(
MaxPooling2D(
(2,2)
)
)

Flatten:

model.add(
Flatten()
)

Output:

model.add(
Dense(
1,
activation="sigmoid"
)
)

Transfer Learning

Instead of training from zero, use a pre-trained model.

Popular models:

VGG16
ResNet
MobileNet
EfficientNet

Computer Vision Projects

Beginner Projects

Image converter
Face detector
Color detector

Intermediate Projects

Object detector
Traffic sign recognition
OCR scanner

Advanced Projects

Autonomous driving system
Medical image diagnosis
Real-time surveillance AI

Practice Exercises

Read and display images using OpenCV.
Resize and crop images.
Convert images to grayscale.
Detect edges.
Build face detection program.
Train a CNN classifier.
Create an object detection project.

Chapter 52: Generative AI and Large Language Models (LLMs)

What is Generative AI?

Generative AI is a branch of Artificial Intelligence that can create new content such as:

Text
Images
Audio
Video
Code

Instead of only predicting results, Generative AI can generate new information based on patterns learned from data.

Example:

Input:

Write a Python program for calculator

Output:

def add(a,b):
    return a+b

Traditional AI vs Generative AI

Traditional AI	Generative AI
Classifies data	Creates new content
Predicts results	Generates output
Example: Spam detection	Example: Chatbot
Uses existing rules	Learns patterns

Examples of Generative AI

Text Generation

Examples:

Chatbots
Writing assistants
Code generation

Image Generation

Examples:

AI artwork
Product designs
Image editing

Audio Generation

Examples:

Voice synthesis
Music generation

Video Generation

Examples:

AI animations
Video creation

What are Large Language Models (LLMs)?

A Large Language Model (LLM) is an AI model trained on massive amounts of text data to understand and generate human-like language.

Examples:

ChatGPT
Claude
Gemini
Llama

How LLMs Work

Basic process:

Large Text Dataset

        ↓

Neural Network Training

        ↓

Language Understanding

        ↓

Text Generation

LLM Training Process

Step 1: Data Collection

The model learns from large datasets:

Books
Articles
Websites
Code

Step 2: Tokenization

Text is converted into tokens.

Example:

Sentence:

Python is powerful

Tokens:

Python
is
powerful

Computer representation:

[1245, 876, 4321]

What is a Token?

A token is a small unit of text.

Examples:

Word:

Python

Token:

Python

Long word:

Programming

May become:

Program + ming

Step 3: Model Training

The model learns:

Word relationships
Grammar
Context
Patterns

Example:

Input:

The capital of France is

Model predicts:

Paris

Transformer Architecture

Modern LLMs use the Transformer architecture.

Introduced in 2017.

Main idea:

Attention Mechanism

Transformer Structure

Input Text

     ↓

Tokenizer

     ↓

Embedding

     ↓

Transformer Layers

     ↓

Output Prediction

What is Attention?

Attention allows the model to focus on important words.

Example:

Sentence:

The animal didn't cross the road because it was tired.

The model understands:

it = animal

because of context.

Transformer Components

1. Embedding Layer

Converts tokens into numbers.

Example:

Python

↓

[0.25,0.87,0.34]

2. Attention Layer

Finds relationships between words.

3. Feed Forward Network

Processes information.

4. Output Layer

Generates next token.

How ChatGPT Works (Simplified)

User Question

      ↓

Tokenization

      ↓

Transformer Model

      ↓

Predict Next Tokens

      ↓

Generated Answer

Text Generation Process

Example:

Input:

Python is

Model predicts:

Python is a

Then:

Python is a programming

Then:

Python is a programming language

This happens repeatedly.

Important LLM Concepts

Parameters

Parameters are learned values inside the model.

More parameters generally mean:

Better understanding
More capability

Example:

Small Model

↓

Large Model

↓

More Parameters

Context Window

Amount of text the model can remember during a conversation.

Example:

Small context:

Few pages

Large context:

Long documents

Fine-Tuning

Training an existing model on a specific dataset.

Example:

General model:

Language AI

Fine-tuned:

Medical AI Assistant

Prompt

A prompt is the instruction given to AI.

Example:

Explain Python loops

Prompt Engineering

What is Prompt Engineering?

Creating effective instructions to get better AI responses.

Poor Prompt

Write code

Problem:

Not enough information.

Better Prompt

Write a Python function that calculates factorial.
Explain each step for beginners.

Prompt Structure

Good prompts contain:

1. Role

Example:

You are a Python teacher.

2. Task

Example:

Explain decorators.

3. Context

Example:

Explain for beginners.

4. Format

Example:

Use examples and code.

Types of Prompts

Zero-Shot Prompting

No example provided.

Example:

Translate English to French:
Hello

Few-Shot Prompting

Provide examples.

Example:

Cat → Animal

Car → Vehicle

Dog →

Chain-of-Thought Prompting

Breaking complex tasks into steps.

Example:

Solve step by step.

Using LLMs with Python

Popular libraries:

OpenAI API
Hugging Face Transformers
LangChain

Hugging Face Transformers Example

Install:

pip install transformers

Import:

from transformers import pipeline

Create Text Generator:

generator=pipeline(
"text-generation"
)

Generate Text:

result=generator(
"Python is",
max_length=50
)

print(result)

Building AI Applications

1. AI Chatbot

Components:

User

↓

Chat Interface

↓

LLM

↓

Response

2. Document Question Answering

Process:

PDF

↓

Extract Text

↓

Create Embeddings

↓

Search Relevant Information

↓

LLM Answer

3. AI Coding Assistant

Features:

Code generation
Bug fixing
Explanation

Retrieval Augmented Generation (RAG)

What is RAG?

RAG combines:

External knowledge
LLM generation

Process:

Question

 ↓

Search Documents

 ↓

Find Relevant Data

 ↓

LLM Generates Answer

LLM Limitations

1. Hallucination

AI may generate incorrect information.

2. Bias

Models may learn unwanted patterns.

3. Data Limitations

Knowledge depends on training data.

4. Cost

Large models require:

Powerful hardware
Computing resources

Generative AI Projects with Python

Beginner

Text generator
AI chatbot
Email writer

Intermediate

PDF question-answer system
AI summarizer
Document analyzer

Advanced

Custom LLM chatbot
RAG application
AI coding assistant

Practice Exercises

Learn tokenization.
Create text generation using Transformers.
Build a simple chatbot.
Practice prompt engineering.
Create a document Q&A system.
Explore RAG architecture.

Chapter 53: Building AI Applications with Python

What are AI Applications?

An AI application is a software system that uses Artificial Intelligence models to solve real-world problems.

Examples:

AI chatbots
Recommendation systems
Image recognition apps
Voice assistants
Document analysis tools

AI Application Architecture

A typical AI application contains:

User Interface

      ↓

Backend Application

      ↓

AI Model / API

      ↓

Database

      ↓

Response

Components of an AI Application

1. Frontend

The part users interact with.

Examples:

Web page
Mobile app
Chat interface

Technologies:

HTML
CSS
JavaScript
React

2. Backend

Handles:

User requests
Business logic
AI communication

Python frameworks:

Flask
Django
FastAPI

3. AI Model

The intelligence layer.

Examples:

Machine Learning model
Deep Learning model
LLM

4. Database

Stores:

User data
Conversations
Documents

Examples:

MySQL
PostgreSQL
MongoDB
Vector databases

AI Development Workflow

Problem Definition

        ↓

Collect Data

        ↓

Prepare Data

        ↓

Build AI Model

        ↓

Test Model

        ↓

Create Application

        ↓

Deploy

Using AI APIs with Python

Many AI applications use APIs instead of training models from scratch.

Examples:

Language models
Image models
Speech models

What is an API?

API allows two applications to communicate.

Example:

Python App

     ↓

AI API

     ↓

AI Response

API Request Example

Install requests:

pip install requests

Python:

import requests


response=requests.get(
"https://api.example.com"
)


print(response.json())

Building a Simple AI Chatbot

Chatbot Architecture

User Message

      ↓

Python Backend

      ↓

Language Model

      ↓

Generated Response

      ↓

User

Basic Chatbot Logic

while True:

    message=input(
    "You: "
    )


    if message=="exit":
        break


    print(
    "AI:",
    "I received your message"
    )

Adding Memory

A chatbot can store previous messages.

Example:

conversation=[

"Hello",

"How are you?"

]

The AI uses previous messages for context.

Building a Document Q&A System

Problem

Users upload documents and ask questions.

Example:

Upload:

Company Policy PDF

Question:

What is the leave policy?

Answer:

AI finds information from PDF

Document Q&A Architecture

PDF File

   ↓

Text Extraction

   ↓

Text Splitting

   ↓

Embeddings

   ↓

Vector Database

   ↓

LLM

   ↓

Answer

Text Extraction

Libraries:

PyPDF

Install:

pip install pypdf

Example:

from pypdf import PdfReader


reader=PdfReader(
"document.pdf"
)


text=""


for page in reader.pages:

    text+=page.extract_text()

Text Splitting

Large documents are divided into smaller chunks.

Example:

Before:

100 page document

After:

Chunk 1

Chunk 2

Chunk 3

Embeddings

What are Embeddings?

Embeddings convert text into numbers representing meaning.

Example:

"Python programming"

↓

[0.23,0.78,0.12]

Similar meanings have similar vectors.

Vector Database

Stores embeddings.

Examples:

ChromaDB
FAISS
Pinecone

Used for:

Similarity search
Document retrieval

Retrieval Augmented Generation (RAG)

RAG improves AI answers using external knowledge.

Process:

User Question

      ↓

Search Database

      ↓

Find Relevant Information

      ↓

Send Context to LLM

      ↓

Generate Answer

AI Agents

What is an AI Agent?

An AI agent is a system that can:

Understand goals
Make decisions
Use tools
Complete tasks

Example:

User:

Book a flight

Agent:

1. Search flights

2. Compare prices

3. Select option

4. Complete booking

AI Agent Components

AI Brain

    ↓

Memory

    ↓

Tools

    ↓

Actions

Tools Used by AI Agents

Examples:

Web search
Calculator
Database
APIs
File systems

LangChain Introduction

What is LangChain?

LangChain is a Python framework for building LLM applications.

Used for:

Chatbots
RAG systems
AI agents

Install:

pip install langchain

LangChain Concepts

Chains

Connect multiple AI steps.

Example:

Question

 ↓

Search

 ↓

AI Response

Agents

AI systems that choose actions.

Memory

Stores conversation history.

Building a Simple FastAPI AI Backend

Install:

pip install fastapi uvicorn

Create API:

from fastapi import FastAPI


app=FastAPI()


@app.get("/")
def home():

    return {
    "message":"AI Application Running"
    }

Run:

uvicorn main:app --reload

AI Application Deployment

Common platforms:

Cloud servers
Containers
Serverless platforms

Docker for AI Applications

Docker packages applications.

Example:

Application

+

Libraries

+

Environment

=

Docker Container

Monitoring AI Applications

Important metrics:

Performance

Response time
Accuracy

User Metrics

Usage
Feedback

Model Metrics

Error rate
Prediction quality

Security in AI Applications

Important areas:

Data Protection

Protect:

User information
Documents

API Security

Use:

Authentication
Rate limits

Prompt Security

Prevent:

Prompt injection
Data leakage

AI Projects Using Python

Beginner Projects

AI chatbot
Text summarizer
Sentiment analyzer

Intermediate Projects

PDF Q&A system
AI customer support bot
Resume analyzer

Advanced Projects

AI agent system
RAG chatbot
Custom AI assistant

Practice Exercises

Create a simple chatbot.
Build a FastAPI AI backend.
Extract text from PDF.
Create embeddings.
Build a simple RAG system.
Create an AI agent workflow.

Chapter 54: Data Science with Python

What is Data Science?

Data Science is the process of collecting, cleaning, analyzing, and interpreting data to extract useful information and make decisions.

It combines:

Programming
Statistics
Machine Learning
Data Visualization
Domain Knowledge

Data Science Workflow

A typical data science project follows:

Problem Definition

        ↓

Data Collection

        ↓

Data Cleaning

        ↓

Exploratory Data Analysis

        ↓

Feature Engineering

        ↓

Model Building

        ↓

Evaluation

        ↓

Deployment

Applications of Data Science

Business

Sales prediction
Customer analysis
Market research

Healthcare

Disease prediction
Patient analysis

Finance

Fraud detection
Risk analysis

Marketing

Customer segmentation
Recommendation systems

Python Libraries for Data Science

NumPy

Used for:

Numerical calculations
Arrays
Mathematical operations

Install:

pip install numpy

Pandas

Used for:

Data handling
Data analysis
Tables

Install:

pip install pandas

Matplotlib

Used for:

Data visualization
Charts

Install:

pip install matplotlib

Seaborn

Used for:

Statistical visualization

Install:

pip install seaborn

NumPy Introduction

Import NumPy

import numpy as np

Creating NumPy Array

Python list:

numbers=[1,2,3,4]

Convert to array:

array=np.array(numbers)

print(array)

Output:

[1 2 3 4]

Why Use NumPy?

Normal Python:

Slow calculations

NumPy:

Fast mathematical operations

Array Operations

Addition

a=np.array([1,2,3])

b=np.array([4,5,6])


print(a+b)

Output:

[5 7 9]

Multiplication

print(a*2)

Output:

[2 4 6]

Creating Special Arrays

Zeros Array

np.zeros(5)

Output:

[0. 0. 0. 0. 0.]

Ones Array

np.ones(5)

Range Array

np.arange(1,10)

Output:

[1 2 3 4 5 6 7 8 9]

Array Shape

Check dimensions:

array.shape

Example:

(3,4)

Meaning:

3 rows

4 columns

Reshaping Arrays

Example:

array.reshape(2,3)

Changes:

Before:

6 values


After:

2 rows × 3 columns

Statistical Operations with NumPy

Mean

Average value:

np.mean(array)

Median

Middle value:

np.median(array)

Standard Deviation

Measures spread:

np.std(array)

Pandas Introduction

What is Pandas?

Pandas provides data structures for data analysis.

Main objects:

Series
DataFrame

Import Pandas

import pandas as pd

Pandas Series

A one-dimensional data structure.

Example:

data=pd.Series(
[10,20,30]
)

print(data)

Pandas DataFrame

A table structure.

Example:

data={

"Name":[
"John",
"Alex"
],

"Age":[
25,
30
]

}


df=pd.DataFrame(data)


print(df)

Output:

Name    Age

John    25

Alex    30

Reading Data Files

CSV File

df=pd.read_csv(
"data.csv"
)

Excel File

df=pd.read_excel(
"data.xlsx"
)

Viewing Data

First rows:

df.head()

Last rows:

df.tail()

Data Information

Check columns:

df.info()

Statistics:

df.describe()

Example output:

mean

min

max

standard deviation

Selecting Columns

Example:

df["Age"]

Selecting Multiple Columns

df[
[
"Name",
"Age"
]
]

Filtering Data

Example:

Find people older than 25:

df[
df["Age"]>25
]

Adding New Column

df["Salary"]=[30000,40000]

Removing Column

df.drop(
"Salary",
axis=1
)

Handling Missing Data

Real datasets contain missing values.

Example:

Name     Age

John     25

Alex     NaN

Find Missing Values

df.isnull()

Remove Missing Values

df.dropna()

Fill Missing Values

df.fillna(0)

Data Cleaning

Common tasks:

Remove duplicates
Fix incorrect values
Handle missing data
Convert data types

Removing Duplicates

df.drop_duplicates()

Data Visualization

Visualization helps understand data patterns.

Matplotlib

Import:

import matplotlib.pyplot as plt

Line Chart

Example:

x=[1,2,3,4]

y=[10,20,15,30]


plt.plot(
x,
y
)

plt.show()

Bar Chart

Used for comparisons.

plt.bar(
x,
y
)

plt.show()

Histogram

Shows distribution.

plt.hist(
data
)

plt.show()

Scatter Plot

Shows relationships.

plt.scatter(
x,
y
)

plt.show()

Seaborn Introduction

Import:

import seaborn as sns

Creating Statistical Charts

Example:

sns.histplot(
df["Age"]
)

Correlation Analysis

Correlation shows relationships between variables.

Example:

Height ↑

Weight ↑

Positive correlation.

Calculate:

df.corr()

Exploratory Data Analysis (EDA)

EDA means understanding data before modeling.

Steps:

Understand Data

↓

Clean Data

↓

Find Patterns

↓

Visualize

↓

Prepare Model

Feature Engineering

Creating useful features from existing data.

Example:

Date:

2026-07-20

Create:

Day

Month

Year

Machine Learning in Data Science

After analysis:

Clean Data

      ↓

Features

      ↓

ML Model

      ↓

Prediction

Data Science Project Example

Customer Churn Prediction

Goal:

Predict customers leaving a company.

Steps:

Collect customer data
Clean data
Analyze patterns
Train ML model
Predict churn

Data Science Tools

Popular tools:

Jupyter Notebook
Google Colab
VS Code
Anaconda

Practice Exercises

Create NumPy arrays.
Perform mathematical operations.
Load CSV using Pandas.
Clean missing data.
Create charts.
Perform EDA on a dataset.
Build a small data analysis project.

Chapter 55: Statistics for Data Science with Python

What is Statistics?

Statistics is the science of collecting, organizing, analyzing, interpreting, and presenting data.

In Data Science, statistics helps us:

Understand data
Find patterns
Make predictions
Validate models
Make decisions

Why Statistics is Important in Data Science?

Machine Learning models learn from data.

Statistics helps answer:

What does the data tell us?
Are results meaningful?
Are patterns real or random?
How accurate is our prediction?

Types of Statistics

Statistics is divided into two main types:

Statistics

     |

     |

----------------

|              |

Descriptive   Inferential
Statistics    Statistics

1. Descriptive Statistics

Describes existing data.

Examples:

Average age
Maximum salary
Data distribution

2. Inferential Statistics

Makes predictions about a larger population using samples.

Examples:

Election surveys
Medical studies
Market research

Population and Sample

Population

Entire group being studied.

Example:

All students in India

Sample

A smaller group selected from population.

Example:

1000 students from India

Data Types in Statistics

1. Numerical Data

Contains numbers.

Examples:

Age
Salary
Height

2. Categorical Data

Contains categories.

Examples:

Gender
City
Product type

Measures of Central Tendency

Central tendency describes the center of data.

Main methods:

Mean
Median
Mode

1. Mean (Average)

Formula:

Mean = Sum of values / Number of values

Example:

Data:

10,20,30

Calculation:

(10+20+30)/3

=20

Python:

import numpy as np


data=[10,20,30]


np.mean(data)

Output:

2. Median

Median is the middle value after sorting.

Example:

10,20,30,40,50

Median:

Python:

np.median(data)

3. Mode

Most frequently occurring value.

Example:

10,20,20,30

Mode:

Python:

from scipy import stats


stats.mode(data)

Measures of Dispersion

Shows how spread out data is.

Methods:

Range
Variance
Standard deviation

1. Range

Formula:

Maximum - Minimum

Example:

10,20,30

Range:

30-10=20

Python:

max(data)-min(data)

2. Variance

Variance measures how far values are from the mean.

High variance:

Data is spread out

Low variance:

Data is close together

Python:

np.var(data)

3. Standard Deviation

Standard deviation is the square root of variance.

Formula:

√Variance

Python:

np.std(data)

Probability Basics

What is Probability?

Probability measures the chance of an event happening.

Formula:

Probability =
Favorable Outcomes /
Total Outcomes

Example:

Coin:

Heads = 1

Tails = 1

Probability of heads:

1/2 = 0.5

Probability Range

Probability always:

0 ≤ Probability ≤ 1

Meaning:

0 = Impossible

1 = Certain

Conditional Probability

Probability of an event happening given another event.

Formula:

P(A|B)

Example:

Probability of rain given clouds.

Bayes Theorem

Bayes theorem updates probability based on new information.

Formula:

P(A|B)=
P(B|A)P(A)
/
P(B)

Used in:

Spam detection
Medical diagnosis
Machine learning

Probability Distributions

A distribution shows how data values are spread.

1. Normal Distribution

Also called:

Gaussian Distribution

Shape:

        *
      *   *
    *       *
  *           *

Examples:

Height
Weight
Exam scores

Normal Distribution Properties

Mean:

Center

Standard deviation:

Spread

2. Binomial Distribution

Used when outcomes are:

Success
Failure

Examples:

Coin toss
Yes/No prediction

3. Poisson Distribution

Used for counting events.

Examples:

Number of customers per hour
Number of calls received

Sampling

Sampling means selecting data from a population.

Types:

Random Sampling

Everyone has equal chance.

Stratified Sampling

Divide population into groups.

Example:

Students:

Class A

Class B

Class C

Hypothesis Testing

Used to check whether an assumption is true.

Example:

Question:

Does a new medicine work?

Hypotheses

Null Hypothesis (H0)

Assumes no change.

Example:

Medicine has no effect

Alternative Hypothesis (H1)

Assumes change exists.

Example:

Medicine improves health

P-Value

P-value tells whether results are significant.

Common rule:

p < 0.05

means:

Reject Null Hypothesis

Confidence Interval

A range of values where the true value is likely found.

Example:

Average salary:

₹50,000 ± ₹5,000

Correlation

Correlation measures relationship between variables.

Range:

-1 to +1

Positive Correlation

Both increase together.

Example:

Study Time ↑

Marks ↑

Negative Correlation

One increases while other decreases.

Example:

Price ↑

Demand ↓

No Correlation

No relationship.

Python Statistical Analysis

Using Pandas

Example:

import pandas as pd


df.describe()

Output:

mean

std

min

max

Correlation in Python

df.corr()

Visualization of Distribution

Histogram:

import matplotlib.pyplot as plt


plt.hist(data)

plt.show()

Box Plot

Shows:

Median
Range
Outliers

Python:

import seaborn as sns


sns.boxplot(
data=data
)

Outliers

Outliers are unusual data points.

Example:

Normal:

10,20,30,40

Outlier:

Detecting Outliers

Using IQR method:

IQR = Q3 - Q1

Values outside range are outliers.

Statistics in Machine Learning

Statistics helps in:

Feature Selection

Finding important variables.

Model Evaluation

Checking performance.

Data Preparation

Understanding patterns.

Real-World Example

House Price Prediction

Data:

Area

Bedrooms

Location

Price

Statistics helps:

Find average prices
Detect unusual houses
Find feature relationships
Prepare ML model

Practice Exercises

Calculate mean, median, and mode.
Find variance and standard deviation.
Create probability examples.
Plot normal distribution.
Calculate correlation.
Detect outliers.
Perform hypothesis testing.

Chapter 56: Big Data and Data Engineering with Python

What is Big Data?

Big Data refers to extremely large and complex datasets that cannot be easily processed using traditional data processing tools.

Examples:

Social media data
Internet search data
Financial transactions
Sensor data
Video data

Why Big Data is Important?

Organizations use Big Data to:

Find patterns
Make predictions
Improve business decisions
Build AI systems

Characteristics of Big Data (5 Vs)

1. Volume

Amount of data.

Example:

TB, PB, EB of data

2. Velocity

Speed at which data is generated.

Example:

Social media posts every second

3. Variety

Different types of data.

Examples:

Text
Images
Videos
Audio
Numbers

4. Veracity

Quality and reliability of data.

Example:

Incorrect data
Duplicate records

5. Value

Useful information extracted from data.

Big Data Architecture

A typical Big Data system:

Data Sources

      ↓

Data Collection

      ↓

Data Storage

      ↓

Data Processing

      ↓

Data Analysis

      ↓

Insights

What is Data Engineering?

Data Engineering is the process of designing systems to collect, store, process, and deliver data.

A Data Engineer builds:

Data pipelines
Databases
Data warehouses
Processing systems

Data Scientist vs Data Engineer

Data Engineer	Data Scientist
Builds data systems	Analyzes data
Creates pipelines	Builds models
Manages storage	Finds insights
Works with infrastructure	Works with statistics

Data Engineering Workflow

Collect Data

     ↓

Store Data

     ↓

Clean Data

     ↓

Transform Data

     ↓

Analyze Data

     ↓

Use Data

Data Pipeline

A data pipeline moves data from source to destination.

Example:

Website Data

     ↓

Pipeline

     ↓

Database

     ↓

Analytics

ETL Process

ETL means:

Extract

Transform

Load

1. Extract

Collect data from sources.

Sources:

Databases
APIs
Files
Sensors

Example:

CSV File

↓

Extract Data

2. Transform

Clean and modify data.

Examples:

Remove duplicates
Convert formats
Handle missing values

3. Load

Store processed data.

Destinations:

Database
Data warehouse
Cloud storage

ETL Example with Python

Extract CSV

import pandas as pd


data=pd.read_csv(
"sales.csv"
)

Transform

Remove missing values:

data=data.dropna()

Load

Save cleaned data:

data.to_csv(
"clean_sales.csv"
)

Data Storage Systems

Traditional Databases

Used for structured data.

Examples:

MySQL
PostgreSQL
Oracle

Data Warehouse

Stores large historical data.

Examples:

Amazon Redshift
Google BigQuery
Snowflake

Data Lake

Stores raw data.

Can contain:

Text
Images
Videos
Logs

SQL for Data Engineering

SQL is used to manage databases.

Creating Table

CREATE TABLE employees(

id INT,

name VARCHAR(50),

salary INT

);

Insert Data

INSERT INTO employees

VALUES

(1,'John',50000);

Read Data

SELECT *

FROM employees;

Filtering Data

SELECT *

FROM employees

WHERE salary > 40000;

Aggregation

Average salary:

SELECT AVG(salary)

FROM employees;

Python with SQL

Install:

pip install sqlalchemy

Connect Database:

from sqlalchemy import create_engine


engine=create_engine(
"database_connection"
)

Read Data:

df=pd.read_sql(
"SELECT * FROM employees",
engine
)

Apache Spark

What is Apache Spark?

Apache Spark is a distributed data processing framework used for Big Data.

Used for:

Large-scale processing
Machine learning
Data analytics

Why Spark?

Traditional processing:

One Computer

↓

Limited Data

Spark:

Many Computers

↓

Huge Data

Spark Architecture

Driver Program

      ↓

Cluster Manager

      ↓

Worker Nodes

      ↓

Data Processing

PySpark

PySpark is Python API for Apache Spark.

Install:

pip install pyspark

Creating Spark Session

from pyspark.sql import SparkSession


spark=SparkSession.builder \
.appName("Example") \
.getOrCreate()

Reading Data in Spark

df=spark.read.csv(
"data.csv"
)

Display Data

df.show()

Data Processing with Spark

Filter:

df.filter(
df.age>25
).show()

Spark Advantages

✅ Handles huge datasets
✅ Fast processing
✅ Distributed computing
✅ Supports machine learning

Distributed Computing

What is Distributed Computing?

Breaking a large task into smaller tasks and running them on multiple computers.

Example:

Large Dataset

      ↓

Split Data

      ↓

Computer 1
Computer 2
Computer 3

      ↓

Combine Results

Cloud Data Engineering

Popular platforms:

AWS

Services:

S3
Redshift
EMR

Google Cloud

Services:

BigQuery
Cloud Storage
Dataflow

Microsoft Azure

Services:

Azure Data Lake
Synapse Analytics

Data Engineering Tools

Workflow Management

Examples:

Apache Airflow

Used for:

Scheduling pipelines
Monitoring jobs

Data Processing

Examples:

Apache Spark
Hadoop

Databases

Examples:

PostgreSQL
MongoDB

Big Data Machine Learning

Machine Learning with Big Data:

Large Dataset

      ↓

Data Processing

      ↓

Feature Engineering

      ↓

ML Model

      ↓

Prediction

Real-World Projects

1. Customer Analytics Pipeline

Steps:

Customer Data

↓

ETL

↓

Database

↓

Dashboard

2. Social Media Analysis

Process:

Posts

↓

Text Processing

↓

Sentiment Analysis

↓

Reports

3. Sales Data Platform

Includes:

Data warehouse
Reports
Predictions

Practice Exercises

Create an ETL pipeline using Pandas.
Connect Python with SQL database.
Write SQL queries.
Install PySpark.
Process large datasets using Spark.
Design a data pipeline architecture.

Chapter 57: Advanced Python Programming

What is Advanced Python?

Advanced Python focuses on powerful features that help developers write:

Faster programs
Cleaner code
Scalable applications
Professional-level software

Topics include:

Decorators
Generators
Iterators
Context managers
Advanced OOP
Multithreading
Multiprocessing
Async programming

1. Advanced Functions in Python

Functions are reusable blocks of code.

Basic function:

def greet(name):
    return "Hello " + name


print(greet("John"))

Output:

Hello John

Function Arguments

Positional Arguments

Order matters.

def add(a,b):
    return a+b


add(5,10)

Keyword Arguments

Specify parameter names.

add(
a=5,
b=10
)

Default Arguments

Provide default values.

def greet(name="User"):
    print(name)


greet()

Output:

User

Variable Length Arguments

*args

Accepts multiple positional arguments.

def total(*numbers):

    return sum(numbers)


print(
total(1,2,3,4)
)

Output:

**kwargs

Accepts multiple keyword arguments.

def info(**data):

    print(data)


info(
name="Alex",
age=25
)

Output:

{'name':'Alex','age':25}

Lambda Functions

A lambda is a small anonymous function.

Normal function:

def square(x):
    return x*x

Lambda:

square=lambda x:x*x


print(square(5))

Output:

Higher-Order Functions

A function that accepts another function.

Example:

def apply(func,value):

    return func(value)


result=apply(
lambda x:x*2,
5
)

print(result)

Output:

Map Function

Applies function to every item.

Example:

numbers=[1,2,3,4]


result=list(
map(
lambda x:x*2,
numbers
)
)

print(result)

Output:

[2,4,6,8]

Filter Function

Filters items.

Example:

numbers=[1,2,3,4,5]


result=list(
filter(
lambda x:x%2==0,
numbers
)
)

print(result)

Output:

[2,4]

Reduce Function

Combines values.

from functools import reduce


result=reduce(
lambda x,y:x+y,
[1,2,3,4]
)

print(result)

Output:

2. Iterators in Python

What is an Iterator?

An iterator is an object that allows sequential access to elements.

Examples:

Lists
Tuples
Strings

Iterable vs Iterator

Iterable:

Can be looped

Iterator:

Produces next value

Creating Iterator

numbers=[1,2,3]


iterator=iter(numbers)


print(next(iterator))

Output:

Next value:

print(next(iterator))

Output:

Custom Iterator

Create your own iterator:

class Count:

    def __init__(self,max):

        self.max=max

        self.current=0


    def __iter__(self):

        return self


    def __next__(self):

        if self.current < self.max:

            value=self.current

            self.current+=1

            return value

        else:

            raise StopIteration

Usage:

for i in Count(5):

    print(i)

Output:

3. Generators in Python

What is a Generator?

A generator produces values one at a time instead of storing all values in memory.

Uses:

Large datasets
Data processing
Streaming

Normal Function

def numbers():

    return [1,2,3,4]

Stores everything.

Generator Function

Uses:

yield

Example:

def numbers():

    yield 1

    yield 2

    yield 3

Using generator:

for n in numbers():

    print(n)

Output:

1
2
3

Generator Advantages

✅ Less memory usage
✅ Faster processing
✅ Useful for large data

Generator Expression

Similar to list comprehension.

List:

[x*x for x in range(5)]

Generator:

(x*x for x in range(5))

4. Decorators in Python

What is a Decorator?

A decorator modifies or extends a function without changing its code.

Example uses:

Logging
Authentication
Timing
Validation

Basic Decorator

def decorator(func):

    def wrapper():

        print("Before function")

        func()

        print("After function")

    return wrapper

Using decorator:

@decorator

def hello():

    print("Hello")

Output:

Before function

Hello

After function

Decorator with Arguments

def decorator(func):

    def wrapper(*args,**kwargs):

        print("Running")

        return func(*args,**kwargs)

    return wrapper

Real Example: Timing Decorator

import time


def timer(func):

    def wrapper():

        start=time.time()

        func()

        end=time.time()

        print(
        end-start
        )

    return wrapper

5. Context Managers

What is Context Manager?

Used to manage resources automatically.

Examples:

Files
Database connections

Without Context Manager

file=open(
"data.txt"
)

file.close()

Problem:

If error occurs, file may not close.

With Context Manager

with open(
"data.txt"
) as file:

    data=file.read()

Automatically closes file.

Creating Custom Context Manager

Using class:

class Demo:

    def __enter__(self):

        print("Start")


    def __exit__(
    self,
    exc_type,
    exc,
    trace
    ):

        print("End")

6. Advanced Object-Oriented Programming

Multiple Inheritance

A class can inherit from multiple classes.

Example:

class A:
    pass


class B:
    pass


class C(A,B):
    pass

Method Resolution Order (MRO)

Determines which method runs first.

Example:

Class A

↓

Class B

↓

Class C

Python follows MRO rules.

Check:

ClassName.mro()

Abstract Classes

Used to create templates.

Example:

from abc import ABC,abstractmethod


class Animal(ABC):

    @abstractmethod

    def sound(self):

        pass

7. Multithreading

What is Threading?

Running multiple tasks inside one program.

Useful for:

File downloading
Network tasks
I/O operations

Thread Example

import threading


def task():

    print("Running Task")


thread=threading.Thread(
target=task
)


thread.start()

8. Multiprocessing

Uses multiple CPU cores.

Useful for:

Heavy calculations
Data processing

Example:

from multiprocessing import Process


def work():

    print("Process Running")


p=Process(
target=work
)


p.start()

Threading vs Multiprocessing

Threading	Multiprocessing
One CPU process	Multiple processes
Good for I/O	Good for CPU tasks
Uses shared memory	Separate memory

9. Asynchronous Programming

What is Async Programming?

Allows programs to perform tasks without waiting.

Used for:

Web applications
APIs
Network requests

Async Function

async def hello():

    print("Hello")

Await

Wait for async operation.

async def main():

    await hello()

Async Example

import asyncio


async def task():

    print("Start")

    await asyncio.sleep(2)

    print("End")


asyncio.run(
task()
)

Real-World Uses of Advanced Python

Web Development

FastAPI
Django

Data Science

Generators for large datasets

AI Development

Async API calls
Parallel processing

Automation

Decorators
Context managers

Practice Exercises

Create a function using *args.
Build custom iterator.
Create generator for large numbers.
Write a logging decorator.
Create custom context manager.
Use threading for tasks.
Create async functions.

Chapter 58: Python Software Development Practices

What are Software Development Practices?

Software development practices are techniques used by professional developers to create:

Clean code
Maintainable applications
Reliable software
Team-friendly projects

A good Python developer not only writes code that works but writes code that is easy to understand and improve.

Professional Python Development Workflow

Problem

   ↓

Design Solution

   ↓

Write Code

   ↓

Test Code

   ↓

Debug Errors

   ↓

Document

   ↓

Deploy

1. Writing Clean Python Code

Clean code means:

Easy to read
Easy to modify
Well organized

Bad Code Example

x=10
y=20
z=x+y
print(z)

Problem:

No meaningful names
Difficult to understand

Clean Code Example

first_number=10

second_number=20


total=first_number+second_number


print(total)

Python Naming Rules

Variables

Use:

student_name

Avoid:

sn

Functions

Use verbs:

Good:

calculate_salary()

Bad:

salary()

Classes

Use PascalCase:

Good:

StudentRecord

2. PEP 8 Style Guide

PEP 8 is Python’s official coding style guide.

It recommends:

Proper spacing
Naming conventions
Code formatting

Indentation

Python uses indentation.

Correct:

if age>=18:

    print("Adult")

Incorrect:

if age>=18:
print("Adult")

Line Length

Recommended:

Maximum 79 characters

Imports

Good:

import os
import math

3. Project Structure

A professional Python project:

my_project/

│
├── main.py
├── requirements.txt
├── README.md
│
├── src/
│   ├── models.py
│   └── functions.py
│
├── tests/
│   └── test_main.py
│
└── data/

4. Virtual Environments

What is a Virtual Environment?

A virtual environment creates an isolated Python environment for a project.

Benefits:

Avoid package conflicts
Manage dependencies
Professional development

Creating Virtual Environment

Command:

python -m venv myenv

Activate Environment

Windows:

myenv\Scripts\activate

Linux/Mac:

source myenv/bin/activate

Install Packages

Example:

pip install pandas

Deactivate

deactivate

5. Package Management

pip

Python package installer.

Install package:

pip install numpy

View Installed Packages

pip list

Requirements File

Stores project dependencies.

Create:

pip freeze > requirements.txt

Example:

numpy==2.0
pandas==2.2

Install Requirements

pip install -r requirements.txt

6. Modules and Packages

Module

A Python file containing code.

Example:

calculator.py

Package

Collection of modules.

Example:

mypackage/

    math.py

    string.py

Creating a Module

File:

# math_tools.py


def add(a,b):

    return a+b

Using Module:

import math_tools


print(
math_tools.add(5,3)
)

7. Exception Handling in Professional Code

Errors should be handled properly.

Basic Exception Handling

try:

    number=int(input("Enter: "))

except ValueError:

    print("Invalid number")

Multiple Exceptions

try:

    result=10/0


except ZeroDivisionError:

    print("Cannot divide by zero")


except Exception:

    print("Error")

Custom Exceptions

Create your own error:

class AgeError(Exception):

    pass

8. Logging in Python

Logging records application events.

Used for:

Debugging
Monitoring
Error tracking

Example:

import logging


logging.basicConfig(
level=logging.INFO
)


logging.info(
"Application started"
)

Output:

INFO: Application started

Logging Levels

DEBUG

Detailed information.

INFO

Normal operation.

WARNING

Something unexpected.

ERROR

A problem occurred.

CRITICAL

Serious failure.

9. Testing in Python

Testing ensures code works correctly.

Popular library:

unittest

and:

pytest

Installing pytest

pip install pytest

Simple Test

File:

test_math.py

Code:

def add(a,b):

    return a+b


def test_add():

    assert add(2,3)==5

Run:

pytest

Types of Testing

Unit Testing

Tests small parts.

Example:

Function test

Integration Testing

Tests multiple components together.

Example:

Database + Application

System Testing

Tests complete application.

10. Debugging Python Code

Debugging means finding and fixing errors.

Using print()

Simple debugging:

print(variable)

Python Debugger

Use:

import pdb

Example:

pdb.set_trace()

Common Errors

Syntax Error

Wrong Python syntax.

Example:

print("Hello"

Runtime Error

Error while running.

Example:

10/0

Logic Error

Program runs but gives wrong output.

Example:

price*0

11. Documentation

Good software needs documentation.

Comments

Explain code:

# Calculate total price
total=price*quantity

Docstrings

Explain functions:

def add(a,b):
    """
    Adds two numbers
    """
    return a+b

README File

Contains:

Project description
Installation steps
Usage instructions

Example:

# My Python Project

Install:

pip install -r requirements.txt

Run:

python main.py

12. Git Version Control

What is Git?

Git tracks changes in code.

Benefits:

Save versions
Collaboration
Backup

Basic Git Commands

Initialize:

git init

Check Status:

git status

Add Files:

git add .

Commit:

git commit -m "First version"

Push:

git push

GitHub

GitHub stores Git projects online.

Used for:

Collaboration
Open source
Portfolio

13. Code Review

Code review means checking another developer’s code.

Benefits:

Find bugs
Improve quality
Share knowledge

Professional Development Tools

Code Editors

Examples:

VS Code
PyCharm

Formatting Tools

Examples:

Black
Autopep8

Code Quality Tools

Examples:

Pylint
Flake8

14. Deployment Basics

After development:

Code

 ↓

Testing

 ↓

Build

 ↓

Deploy

 ↓

Monitor

Python Deployment Options

Examples:

Cloud servers
Docker
Web hosting platforms

Real Professional Python Project Example

Structure:

AI_Project/

├── app.py

├── requirements.txt

├── README.md

├── config.py

├── database/

├── models/

├── tests/

└── logs/

Practice Exercises

Create a professional Python project structure.
Create and use virtual environments.
Write a requirements.txt file.
Build a module and package.
Add exception handling.
Write unit tests.
Create Git repository.
Write project documentation.

Chapter 59: Python Web Development

What is Web Development?

Web Development is the process of creating websites and web applications that run on browsers.

Examples:

Online shopping websites
Social media platforms
Banking applications
AI web applications

Python is widely used for backend web development.

Types of Web Development

Web Development

       |

-------------------

|                 |

Frontend        Backend

1. Frontend Development

The part users see and interact with.

Technologies:

HTML
CSS
JavaScript
React

Examples:

Buttons
Forms
Pages
Design

2. Backend Development

The server-side logic.

Responsible for:

Processing requests
Database operations
Authentication
APIs

Python frameworks:

Flask
Django
FastAPI

How a Website Works

User Browser

      ↓

HTTP Request

      ↓

Web Server

      ↓

Python Application

      ↓

Database

      ↓

HTTP Response

      ↓

Browser

HTTP Basics

What is HTTP?

HTTP is a communication protocol used between browsers and servers.

Example:

Browser

↓

Request

↓

Server

↓

Response

HTTP Methods

GET

Used to retrieve data.

Example:

View profile

POST

Used to send data.

Example:

Submit login form

PUT

Updates existing data.

Example:

Update profile

DELETE

Removes data.

Example:

Delete account

HTTP Status Codes

200

Success

404

Page not found

500

Server error

Web Frameworks in Python

A framework provides tools to build web applications faster.

Popular Python frameworks:

1. Flask

A lightweight web framework.

Used for:

Small applications
APIs
Prototypes

2. Django

A complete web framework.

Used for:

Large websites
Enterprise applications

3. FastAPI

Modern framework for APIs.

Used for:

AI applications
High-performance APIs

Flask Introduction

Install:

pip install flask

Creating First Flask Application

Create:

app.py

Code:

from flask import Flask


app=Flask(__name__)


@app.route("/")
def home():

    return "Hello Python Web"


app.run()

Run:

python app.py

Open:

http://localhost:5000

Flask Routes

Routes define URLs.

Example:

@app.route("/about")
def about():

    return "About Page"

Visit:

/about

Dynamic Routes

Example:

@app.route("/user/<name>")
def user(name):

    return "Hello "+name

URL:

/user/John

Output:

Hello John

HTML Templates in Flask

Create folder:

templates/

File:

index.html

HTML:

<h1>
Welcome
</h1>

Python:

from flask import render_template


@app.route("/")
def home():

    return render_template(
    "index.html"
    )

Forms in Flask

HTML:

<form method="POST">

<input name="username">

<button>
Submit
</button>

</form>

Python:

from flask import request


@app.route(
"/login",
methods=["POST"]
)

def login():

    username=request.form["username"]

    return username

Flask API Development

API returns data instead of HTML.

Example:

from flask import jsonify


@app.route("/api")

def api():

    return jsonify(
    {
    "name":"Python",
    "level":"Advanced"
    }
    )

Response:

{
"name":"Python",
"level":"Advanced"
}

Django Introduction

What is Django?

Django is a full-stack Python web framework.

Features:

Authentication
Admin panel
Database support
Security

Installing Django

pip install django

Creating Django Project

Command:

django-admin startproject mysite

Project structure:

mysite/

├── manage.py

├── settings.py

├── urls.py

└── views.py

Running Django Server

python manage.py runserver

Django Apps

Large projects are divided into apps.

Example:

Website

 |

 |-- Users App

 |-- Products App

 |-- Payments App

Creating Django App

python manage.py startapp users

Django Views

Views handle requests.

Example:

from django.http import HttpResponse


def home(request):

    return HttpResponse(
    "Hello Django"
    )

Django URLs

Connect URL with views.

path(
"",
views.home
)

Django Models

Models define database tables.

Example:

from django.db import models


class Student(
models.Model
):

    name=models.CharField(
    max_length=100
    )

    age=models.IntegerField()

Database Migration

Create database tables:

python manage.py makemigrations

Then:

python manage.py migrate

Django Admin Panel

Django provides built-in admin interface.

Create admin user:

python manage.py createsuperuser

FastAPI Introduction

What is FastAPI?

FastAPI is a modern Python framework for building APIs.

Advantages:

Very fast
Automatic documentation
Easy for AI applications

Install FastAPI

pip install fastapi uvicorn

Creating FastAPI App

from fastapi import FastAPI


app=FastAPI()


@app.get("/")
def home():

    return {
    "message":"Hello FastAPI"
    }

Run FastAPI

uvicorn main:app --reload

API Endpoint Example

@app.get("/users/{id}")

def user(id:int):

    return {
    "user_id":id
    }

Database Integration

Web applications need databases.

Popular databases:

MySQL
PostgreSQL
SQLite
MongoDB

Using SQLite with Python

Built into Python:

import sqlite3

Connect:

connection=sqlite3.connect(
"database.db"
)

ORM Concept

ORM allows developers to use Python objects instead of SQL directly.

Examples:

Django ORM
SQLAlchemy

SQLAlchemy Example

Install:

pip install sqlalchemy

Create model:

from sqlalchemy import Column,Integer,String

Authentication in Web Applications

Authentication verifies users.

Examples:

Login
Signup
Password reset

Security Practices

Important:

Password hashing
Input validation
HTTPS
Protection against attacks

Web APIs

API allows applications to communicate.

Example:

Mobile App

      ↓

Python API

      ↓

Database

REST API Principles

REST uses:

URLs
HTTP methods
JSON responses

Example:

GET:

/api/products

POST:

/api/products

Building Projects

Beginner Projects

Personal website
Blog application
Todo app

Intermediate Projects

E-commerce website
REST API
User authentication system

Advanced Projects

AI chatbot web app
Social media platform
Enterprise web application

Practice Exercises

Create Flask application.
Create HTML templates.
Build REST API.
Create Django project.
Connect database.
Build FastAPI API.
Create authentication system.

What is Automation?

Automation means using programs to perform tasks automatically without manual effort.

Python is one of the most popular languages for automation because it is:

Easy to learn
Powerful
Has many libraries
Works with files, websites, APIs, and systems

Why Use Python Automation?

Automation helps to:

Save time
Reduce human errors
Increase productivity
Handle repetitive tasks

Examples of Automation

Office Automation

Create reports
Process Excel files
Send emails

File Automation

Rename files
Move files
Backup data

Web Automation

Scrape websites
Fill forms
Test websites

System Automation

Monitor servers
Run scripts
Manage files

Automation Workflow

Identify Task

      ↓

Write Python Script

      ↓

Test Script

      ↓

Schedule Automation

      ↓

Run Automatically

Python Automation Libraries

1. os

Used for operating system tasks.

Examples:

Files
Folders
Paths

2. shutil

Used for file operations.

Examples:

Copy files
Move files

3. pathlib

Modern file handling library.

4. requests

Used for web requests.

5. BeautifulSoup

Used for web scraping.

6. Selenium

Used for browser automation.

7. schedule

Used for task scheduling.

File Automation

Working with Files

Python can:

Create files
Read files
Update files
Delete files

Reading a File

file=open(
"data.txt",
"r"
)

content=file.read()

print(content)

file.close()

Writing a File

file=open(
"output.txt",
"w"
)

file.write(
"Hello Python"
)

file.close()

Better File Handling

Using context manager:

with open(
"data.txt"
) as file:

    content=file.read()

Creating Folders

Using os:

import os


os.mkdir(
"NewFolder"
)

Checking File Exists

os.path.exists(
"data.txt"
)

Listing Files

import os


files=os.listdir(
"."
)

print(files)

Renaming Files Automatically

import os


os.rename(
"old.txt",
"new.txt"
)

Moving Files

Using shutil:

import shutil


shutil.move(
"file.txt",
"folder/"
)

Copying Files

shutil.copy(
"file.txt",
"backup/"
)

Automatic File Organizer Project

Goal:

Arrange files automatically.

Example:

Before:

Downloads/

photo.jpg

report.pdf

song.mp3

After:

Downloads/

Images/

Documents/

Music/

File Organizer Code

import os
import shutil


files=os.listdir(
"Downloads"
)


for file in files:

    if file.endswith(".jpg"):

        shutil.move(
        "Downloads/"+file,
        "Downloads/Images"
        )

Working with Excel Automation

Library:

openpyxl

Install:

pip install openpyxl

Reading Excel File

import openpyxl


workbook=openpyxl.load_workbook(
"data.xlsx"
)


sheet=workbook.active

Writing Excel Data

sheet["A1"]="Name"

sheet["B1"]="Age"


workbook.save(
"new.xlsx"
)

CSV Automation

Using Pandas:

import pandas as pd


data=pd.read_csv(
"sales.csv"
)

Generate Reports Automatically

Example:

Sales Data

↓

Analysis

↓

Excel Report

↓

Email Report

Web Scraping

What is Web Scraping?

Web scraping extracts data from websites automatically.

Examples:

Product prices
News articles
Weather data

BeautifulSoup

Install:

pip install beautifulsoup4

Basic Scraping Example

import requests

from bs4 import BeautifulSoup


url="https://example.com"


response=requests.get(
url
)


soup=BeautifulSoup(
response.text,
"html.parser"
)


print(
soup.title.text
)

Extract Links

links=soup.find_all(
"a"
)


for link in links:

    print(
    link.get("href")
    )

Web Automation with Selenium

What is Selenium?

Selenium controls a web browser using Python.

Used for:

Testing websites
Automating forms
Browser tasks

Install Selenium

pip install selenium

Open Browser

from selenium import webdriver


browser=webdriver.Chrome()


browser.get(
"https://google.com"
)

Find Element

search=browser.find_element(
"id",
"search"
)

Enter Text

search.send_keys(
"Python"
)

Click Button

button.click()

Email Automation

Python can send emails automatically.

Library:

smtplib

Sending Email Example

import smtplib


server=smtplib.SMTP(
"smtp.gmail.com",
587
)


server.starttls()


server.login(
"email",
"password"
)


server.sendmail(
"from",
"to",
"Hello"
)

Email Automation Uses

Daily reports
Notifications
Alerts
Marketing emails

PDF Automation

Library:

PyPDF

Install:

pip install pypdf

Extract PDF Text

from pypdf import PdfReader


reader=PdfReader(
"file.pdf"
)


text=reader.pages[0].extract_text()

Image Automation

Library:

Pillow

Install:

pip install pillow

Resize Image

from PIL import Image


image=Image.open(
"photo.jpg"
)


image.resize(
(500,500)
)

Task Scheduling

Automation often needs to run at specific times.

Examples:

Daily backup
Weekly report
Monthly email

Schedule Library

Install:

pip install schedule

Example:

import schedule
import time


def job():

    print(
    "Running task"
    )


schedule.every().day.do(
job
)


while True:

    schedule.run_pending()

    time.sleep(1)

Automation with APIs

Python can communicate with services.

Example:

import requests


response=requests.get(
"https://api.example.com/data"
)


data=response.json()

System Monitoring Automation

Python can monitor:

CPU usage
Memory
Disk space

Library:

psutil

Install:

pip install psutil

Example:

import psutil


print(
psutil.cpu_percent()
)

Automation Project Ideas

Beginner Projects

File organizer
Password generator
Automatic calculator
PDF reader

Intermediate Projects

Web scraper
Email automation system
Excel report generator
Website testing bot

Advanced Projects

AI automation assistant
Business workflow automation
Server monitoring system
Automated data pipeline

Best Practices for Automation

1. Handle Errors

Use:

try:
    pass

except:
    pass

2. Add Logging

Track:

Success
Failures
Errors

3. Secure Credentials

Do not write passwords directly.

Use:

Environment variables
Secret managers

4. Test Before Automation

Always test scripts before running automatically.

Practice Exercises

Create automatic file organizer.
Build Excel report generator.
Scrape website data.
Automate browser tasks.
Send automated emails.
Create scheduled scripts.
Build a complete automation workflow.

Chapter 61: Python for Cyber Security

What is Cyber Security?

Cyber Security is the practice of protecting computers, networks, applications, and data from unauthorized access, attacks, and damage.

Python is widely used in cyber security because it helps automate security tasks and analyze systems.

Why Python is Used in Cyber Security?

Python is useful because:

Easy to write scripts
Large security library ecosystem
Good for automation
Works with networks
Supports data analysis

Areas of Cyber Security

Cyber Security

       |

----------------------------

|            |             |

Network    Application   Data
Security   Security      Security

Cyber Security Roles

Security Analyst

Works on:

Monitoring threats
Investigating incidents

Penetration Tester

Tests systems for vulnerabilities with permission.

Security Engineer

Builds secure systems.

Ethical Hacker

Finds weaknesses to improve security.

Important Security Concepts

1. Confidentiality

Protecting information from unauthorized access.

Example:

Only authorized users can read data

2. Integrity

Ensuring data is not changed incorrectly.

Example:

File remains unchanged

3. Availability

Systems should remain accessible.

Example:

Website stays online

CIA Triad

The foundation of security:

Confidentiality

Integrity

Availability

Python Security Applications

Python is used for:

Log analysis
Security automation
Network monitoring
Vulnerability scanning
Malware analysis
Digital forensics

Python Security Libraries

1. Socket

Used for network programming.

Built into Python.

2. Requests

Used for HTTP communication.

Install:

pip install requests

3. Scapy

Used for network packet analysis.

Install:

pip install scapy

4. Cryptography

Used for encryption.

Install:

pip install cryptography

5. Paramiko

Used for SSH automation.

Install:

pip install paramiko

Network Basics

What is a Network?

A network connects computers to share information.

Examples:

Internet
Local networks
Cloud networks

IP Address

An IP address identifies a device on a network.

Example:

192.168.1.10

Port

A port identifies a service running on a device.

Examples:

HTTP  → 80

HTTPS → 443

SSH   → 22

Client and Server Model

Client

  ↓ Request

Server

  ↓ Response

Client

Socket Programming

A socket allows communication between programs.

Creating a Socket

import socket


s=socket.socket(
socket.AF_INET,
socket.SOCK_STREAM
)

Connecting to Server

s.connect(
("example.com",80)
)

Network Information

Getting computer hostname:

import socket


hostname=socket.gethostname()


print(hostname)

Checking IP Address

ip=socket.gethostbyname(
"example.com"
)

print(ip)

Encryption Basics

What is Encryption?

Encryption converts readable data into protected data.

Example:

Original Data

      ↓

Encryption

      ↓

Encrypted Data

Types of Encryption

Symmetric Encryption

Same key used for:

Encryption
Decryption

Example:

AES

Asymmetric Encryption

Uses:

Public key
Private key

Example:

RSA

Hashing

Hashing converts data into a fixed-length value.

Example:

Password

↓

Hash

↓

Stored Value

Hash Example in Python

import hashlib


text="password"


hash_value=hashlib.sha256(
text.encode()
).hexdigest()


print(hash_value)

Password Security

Good password practices:

Use strong passwords
Store hashed passwords
Use multi-factor authentication

Password Hashing

Never store:

password123

Store:

hash_value

Secure Password Example

Library:

from werkzeug.security import generate_password_hash

Log Analysis with Python

Security teams analyze logs to find:

Failed logins
Suspicious activity
Errors

Example log:

FAILED LOGIN user1

FAILED LOGIN user2

SUCCESS LOGIN user3

Reading Log Files

with open(
"server.log"
) as file:

    logs=file.readlines()

Finding Failed Attempts

for log in logs:

    if "FAILED" in log:

        print(log)

File Integrity Checking

Security systems check whether files have changed.

Using hashes:

import hashlib


def file_hash(filename):

    data=open(filename,"rb").read()

    return hashlib.sha256(
    data
    ).hexdigest()

Security Automation

Python can automate:

Security reports
Log monitoring
Alerts
Compliance checks

Example Security Monitoring Workflow

System Logs

      ↓

Python Script

      ↓

Analyze Events

      ↓

Generate Alert

Vulnerability Scanning Concepts

Security scanners check for:

Weak configurations
Missing updates
Security issues

Python can help automate authorized security checks.

Network Packet Analysis

Packets contain network information.

Example:

Source IP

Destination IP

Protocol

Data

Scapy Introduction

Scapy helps analyze packets.

Example:

from scapy.all import *


packet=IP()


print(packet)

Digital Forensics

Digital forensics investigates digital evidence.

Python helps with:

File analysis
Metadata extraction
Log analysis

Metadata Example

Images may contain:

Camera information
Date
Location information

Python can analyze metadata.

Security Tools Built with Python

Examples:

Network monitors
Log analyzers
Security dashboards
Automation scripts

Secure Coding Practices

1. Validate Input

Never trust user input.

2. Protect Secrets

Avoid:

password="12345"

Use environment variables.

3. Handle Errors Safely

Avoid exposing sensitive information.

4. Keep Libraries Updated

Old libraries may contain vulnerabilities.

Cyber Security Projects Using Python

Beginner Projects

Password strength checker
File hash checker
Log analyzer

Intermediate Projects

Network monitor
Security report generator
Encryption tool

Advanced Projects

Security automation platform
Threat monitoring system
Digital forensic analyzer

Practice Exercises

Create a file hashing program.
Build a password security checker.
Analyze server logs.
Create a network information tool.
Learn encryption with Python.
Build security monitoring scripts.

Chapter 62: Python Cloud Computing

What is Cloud Computing?

Cloud Computing is the delivery of computing services over the internet instead of using local computers.

Cloud services include:

Servers
Storage
Databases
Networking
Software
Artificial Intelligence services

Why Use Python in Cloud Computing?

Python is popular in cloud development because:

Simple syntax
Excellent automation support
Many cloud libraries
Used in AI and data applications

Cloud Computing Architecture

User

 ↓

Internet

 ↓

Cloud Provider

 ↓

Servers + Storage + Database

 ↓

Application

Major Cloud Providers

1. Amazon Web Services (AWS)

Provides:

Compute
Storage
Databases
AI services

2. Google Cloud Platform (GCP)

Provides:

Data analytics
AI tools
Cloud infrastructure

3. Microsoft Azure

Provides:

Enterprise cloud services
Databases
AI platforms

Cloud Service Models

1. IaaS (Infrastructure as a Service)

Provides:

Virtual machines
Networks
Storage

Example:

Rent a server online

2. PaaS (Platform as a Service)

Provides:

Development environment
Deployment tools

Example:

Deploy Python application easily

3. SaaS (Software as a Service)

Provides ready-to-use software.

Examples:

Email
Online applications

Cloud Deployment Models

Public Cloud

Available to everyone.

Example:

AWS

Private Cloud

Used by one organization.

Hybrid Cloud

Combination of public and private cloud.

Cloud Computing Workflow with Python

Python Application

       ↓

Cloud API

       ↓

Cloud Service

       ↓

Result

Python Cloud Libraries

AWS

Library:

boto3

Install:

pip install boto3

Google Cloud

Library:

google-cloud

Install:

pip install google-cloud

Azure

Library:

azure-sdk

Install:

pip install azure-sdk

AWS with Python

What is AWS?

Amazon Web Services is one of the largest cloud platforms.

Popular AWS services:

EC2
S3
Lambda
RDS
DynamoDB

AWS S3 Storage

What is S3?

S3 stores files in the cloud.

Examples:

Images
Videos
Backups
Documents

Upload File to S3

Example:

import boto3


s3=boto3.client(
"s3"
)


s3.upload_file(
"image.jpg",
"my-bucket",
"image.jpg"
)

Download File from S3

s3.download_file(
"my-bucket",
"image.jpg",
"download.jpg"
)

AWS EC2

What is EC2?

EC2 provides virtual servers.

You can run:

Websites
APIs
Applications

Python Connecting to EC2

Using boto3:

import boto3


ec2=boto3.client(
"ec2"
)


instances=ec2.describe_instances()


print(instances)

AWS Lambda

What is Lambda?

A serverless computing service.

You run code without managing servers.

Lambda Function Example

def lambda_handler(event,context):

    return {

    "message":"Hello Cloud"

    }

Google Cloud with Python

Google Cloud Services

Examples:

Cloud Storage
BigQuery
Cloud Functions
Compute Engine

Google Cloud Storage

Install:

pip install google-cloud-storage

Upload File Example

from google.cloud import storage


client=storage.Client()


bucket=client.bucket(
"my-bucket"
)


blob=bucket.blob(
"file.txt"
)


blob.upload_from_filename(
"file.txt"
)

Google BigQuery

Used for:

Big data analysis
Data warehouses

Python:

from google.cloud import bigquery


client=bigquery.Client()

Microsoft Azure with Python

Azure services:

Virtual Machines
Blob Storage
Functions
Databases

Azure Storage Example

Install:

pip install azure-storage-blob

Serverless Computing

What is Serverless?

Developers run applications without managing servers.

Benefits:

Automatic scaling
Pay only when used
Easy deployment

Serverless Platforms

Examples:

AWS Lambda
Google Cloud Functions
Azure Functions

Cloud APIs

APIs allow Python programs to communicate with cloud services.

Example:

Python Script

      ↓

Cloud API

      ↓

Cloud Service

Environment Variables

Cloud applications store secrets safely.

Example:

import os


api_key=os.environ.get(
"API_KEY"
)

Cloud Databases

Popular options:

SQL Databases

MySQL
PostgreSQL

NoSQL Databases

MongoDB
DynamoDB

Python with Cloud Database

Example:

import sqlite3


connection=sqlite3.connect(
"database.db"
)

Deploying Python Applications

Deployment means making an application available online.

Deployment Process

Write Code

    ↓

Test Application

    ↓

Create Cloud Server

    ↓

Upload Code

    ↓

Run Application

Deploying Flask Application

Example platforms:

AWS Elastic Beanstalk
Google App Engine
Azure App Service

Docker and Cloud

What is Docker?

Docker packages applications with dependencies.

Example:

Application

+

Python

+

Libraries

=

Container

Dockerfile Example

FROM python:3.12

COPY app.py .

RUN pip install flask

CMD ["python","app.py"]

Cloud Automation with Python

Python can automate:

Creating servers
Managing storage
Monitoring systems
Deploying applications

Cloud Monitoring

Monitor:

CPU usage
Memory
Errors
Application performance

Artificial Intelligence in Cloud

Cloud AI services provide:

Machine learning APIs
Speech recognition
Image recognition
Language processing

Real-World Cloud Projects

Beginner Projects

Upload files to cloud storage
Create cloud backup script
Build simple cloud API

Intermediate Projects

Deploy Flask application
Create serverless function
Build cloud database application

Advanced Projects

Cloud-based AI application
Data processing pipeline
Automated cloud infrastructure system

Practice Exercises

Learn AWS boto3 basics.
Upload files to cloud storage.
Create serverless functions.
Deploy Python web application.
Connect Python with cloud databases.
Automate cloud resources.

Chapter 63: Python Artificial Intelligence and Deep Learning

What is Artificial Intelligence (AI)?

Artificial Intelligence is the field of computer science that enables machines to perform tasks that normally require human intelligence.

Examples:

Voice assistants
Image recognition
Self-driving cars
Recommendation systems
Chatbots

Why Python is Used for AI?

Python is the most popular AI programming language because:

Simple syntax
Huge ecosystem
Powerful libraries
Strong community support

AI vs Machine Learning vs Deep Learning

Artificial Intelligence

          ↓

Machine Learning

          ↓

Deep Learning

1. Artificial Intelligence (AI)

AI is the broad field of creating intelligent machines.

Examples:

Robots
Expert systems
AI assistants

2. Machine Learning (ML)

Machine Learning allows computers to learn patterns from data.

Example:

Data

 ↓

Learning Algorithm

 ↓

Prediction

3. Deep Learning (DL)

Deep Learning uses artificial neural networks inspired by the human brain.

Used for:

Images
Speech
Language
Complex predictions

AI Workflow

Collect Data

      ↓

Clean Data

      ↓

Train Model

      ↓

Evaluate Model

      ↓

Deploy AI System

AI Libraries in Python

Machine Learning

Scikit-learn

Deep Learning

TensorFlow
Keras
PyTorch

Data Processing

NumPy
Pandas

Visualization

Matplotlib

Installing AI Libraries

Scikit-learn:

pip install scikit-learn

TensorFlow:

pip install tensorflow

PyTorch:

pip install torch

Machine Learning Basics

What is a Model?

A model learns relationships from data.

Example:

Input:

House Area

Bedrooms

Output:

House Price

Types of Machine Learning

Machine Learning

       |

---------------------

|          |         |

Supervised Unsupervised Reinforcement

1. Supervised Learning

Uses labeled data.

Example:

Input → Output

Applications:

Price prediction
Spam detection

2. Unsupervised Learning

Finds patterns without labels.

Examples:

Customer grouping
Data clustering

3. Reinforcement Learning

Agent learns through rewards and penalties.

Examples:

Game AI
Robots

Neural Networks

What is a Neural Network?

A neural network is a computing system inspired by the human brain.

Structure:

Input Layer

      ↓

Hidden Layers

      ↓

Output Layer

Neuron Concept

A neuron receives:

Input values
Weights
Bias

Then produces output.

Formula:

Output = Activation(Input × Weight + Bias)

Activation Functions

Activation functions decide neuron output.

Common functions:

ReLU

Most common in deep learning.

Formula:

max(0,x)

Sigmoid

Output between:

0 and 1

Used for:

Binary classification

Softmax

Used for:

Multiple class classification

Deep Learning Frameworks

TensorFlow

Created by Google.

Used for:

Neural networks
Production AI systems

PyTorch

Created by Meta.

Used for:

Research
Deep learning development

TensorFlow Basics

Import:

import tensorflow as tf

Check version:

print(tf.__version__)

Creating a Simple Neural Network

Using Keras:

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense


model=Sequential()


model.add(
Dense(
10,
activation="relu"
)
)


model.add(
Dense(
1
)
)

Compiling Model

model.compile(
optimizer="adam",
loss="mse"
)

Training Model

model.fit(
X_train,
y_train,
epochs=10
)

Making Predictions

prediction=model.predict(
X_test
)

PyTorch Basics

Import:

import torch

Create Tensor:

x=torch.tensor(
[1,2,3]
)

print(x)

Creating Neural Network in PyTorch

import torch.nn as nn


class Network(nn.Module):

    def __init__(self):

        super().__init__()

        self.layer=nn.Linear(
        10,
        1
        )


    def forward(self,x):

        return self.layer(x)

Computer Vision

What is Computer Vision?

Computer vision allows computers to understand images and videos.

Applications:

Face recognition
Medical imaging
Object detection

Image Processing with Python

Library:

OpenCV

Install:

pip install opencv-python

Reading Image

import cv2


image=cv2.imread(
"photo.jpg"
)


cv2.imshow(
"Image",
image
)

Image Operations

Python can perform:

Resize images
Detect objects
Remove noise
Recognize faces

Convolutional Neural Networks (CNN)

CNNs are specialized neural networks for images.

Used for:

Image classification
Object detection

Structure:

Image

 ↓

Convolution

 ↓

Pooling

 ↓

Classification

Natural Language Processing (NLP)

What is NLP?

NLP allows computers to understand human language.

Applications:

Chatbots
Translation
Sentiment analysis

NLP Libraries

NLTK

Natural Language Toolkit.

Install:

pip install nltk

spaCy

Used for production NLP.

Install:

pip install spacy

Text Processing Steps

Text

 ↓

Tokenization

 ↓

Cleaning

 ↓

Feature Extraction

 ↓

Model

 ↓

Prediction

Tokenization

Breaking text into smaller parts.

Example:

Sentence:

Python is powerful

Tokens:

Python

is

powerful

Sentiment Analysis

Determines emotion in text.

Example:

Input:

"I love this product"

Output:

Positive

Large Language Models (LLMs)

LLMs are AI models trained on huge text datasets.

Examples:

Chatbots
AI assistants
Text generation systems

Transformers

Modern AI models use transformer architecture.

Used in:

Language models
Translation
Image AI

AI Model Training Process

Collect Dataset

      ↓

Prepare Data

      ↓

Create Model

      ↓

Train Model

      ↓

Evaluate

      ↓

Deploy

Model Evaluation

Metrics:

Accuracy

Correct predictions percentage.

Precision

How many predicted positives are correct.

Recall

How many actual positives are found.

F1 Score

Balance between precision and recall.

AI Deployment

AI models can be deployed using:

Flask
FastAPI
Cloud platforms
Docker

Real-World AI Projects

Beginner Projects

Image classifier
Spam detector
Sentiment analyzer

Intermediate Projects

Face recognition system
Recommendation system
Chatbot

Advanced Projects

AI assistant
Computer vision platform
Deep learning application

Practice Exercises

Build a machine learning model.
Create a neural network.
Train an image classifier.
Perform text classification.
Build a simple chatbot.
Deploy an AI model using FastAPI.

Chapter 64: Python Data Science Projects and Portfolio Building

What is a Data Science Portfolio?

A Data Science Portfolio is a collection of projects that demonstrate your skills in:

Python programming
Data analysis
Machine learning
Artificial intelligence
Problem-solving

A strong portfolio helps you:

Get jobs
Apply for internships
Show practical experience
Build credibility

Data Science Workflow

A professional data science project follows:

Problem Definition

        ↓

Data Collection

        ↓

Data Cleaning

        ↓

Exploratory Data Analysis

        ↓

Feature Engineering

        ↓

Model Building

        ↓

Model Evaluation

        ↓

Deployment

        ↓

Documentation

Step 1: Problem Definition

Before writing code, understand the problem.

Example:

Business Problem:

Why are customers leaving our service?

Data Science Goal:

Predict customer churn

Step 2: Data Collection

Sources:

CSV files
Databases
APIs
Web scraping
Sensors

Example:

import pandas as pd


data=pd.read_csv(
"customers.csv"
)

Step 3: Data Cleaning

Real-world data contains:

Missing values
Duplicate records
Wrong formats
Outliers

Checking Missing Values

data.isnull().sum()

Removing Missing Values

data.dropna()

Removing Duplicates

data.drop_duplicates()

Step 4: Exploratory Data Analysis (EDA)

EDA helps understand data.

Used for:

Finding patterns
Discovering relationships
Detecting problems

Basic Statistics

data.describe()

Data Visualization

Libraries:

Matplotlib
Seaborn

Example:

import matplotlib.pyplot as plt


plt.hist(
data["age"]
)

plt.show()

Step 5: Feature Engineering

Feature engineering creates useful input variables.

Example:

Original:

Date = 20-07-2026

Create:

Day

Month

Year

Encoding Categorical Data

Example:

Before:

City

Delhi

Mumbai

After:

Delhi = 0

Mumbai = 1

Python:

pd.get_dummies(data)

Step 6: Splitting Data

Machine learning needs:

Training data
Testing data

Example:

from sklearn.model_selection import train_test_split


X_train,X_test,y_train,y_test=train_test_split(
X,
y,
test_size=0.2
)

Step 7: Model Building

Choose algorithm based on problem.

Regression Projects

Predict numbers.

Examples:

House price
Sales prediction

Algorithms:

Linear Regression
Random Forest
Gradient Boosting

Classification Projects

Predict categories.

Examples:

Spam detection
Fraud detection

Algorithms:

Logistic Regression
Decision Tree
SVM

Clustering Projects

Find groups.

Examples:

Customer segmentation

Algorithm:

K-Means

Example Machine Learning Model

from sklearn.linear_model import LinearRegression


model=LinearRegression()


model.fit(
X_train,
y_train
)

Step 8: Model Evaluation

Different problems use different metrics.

Regression Metrics

Mean Absolute Error

MAE

Mean Squared Error

MSE

R² Score

Measures model performance.

Classification Metrics

Accuracy

Correct predictions.

Precision

Correct positive predictions.

Recall

Detected actual positives.

F1 Score

Combination of precision and recall.

Step 9: Model Deployment

A trained model can become an application.

Deployment options:

Flask
FastAPI
Streamlit
Cloud platforms

Data Science Project Ideas

Project 1: Sales Analysis Dashboard

Goal:

Analyze business sales.

Skills:

Pandas
Visualization
Data cleaning

Features:

Monthly sales trends
Top products
Customer analysis

Project 2: Customer Churn Prediction

Goal:

Predict customers likely to leave.

Skills:

Classification
Feature engineering
Model evaluation

Project 3: House Price Prediction

Goal:

Predict property prices.

Skills:

Regression
Data preprocessing
Model deployment

Project 4: Spam Email Detection

Goal:

Classify emails as spam or normal.

Skills:

NLP
Text processing
Classification

Project 5: Recommendation System

Goal:

Recommend products or movies.

Skills:

Machine learning
Similarity algorithms

Project 6: Sentiment Analysis

Goal:

Analyze customer opinions.

Skills:

NLP
Text classification

Project 7: Image Classification

Goal:

Classify images.

Skills:

Deep learning
CNN
Computer vision

Creating a GitHub Portfolio

What is GitHub?

GitHub stores and shares code projects.

A professional portfolio should include:

Project code
Documentation
Results
Screenshots
Explanation

Project Folder Structure

project_name/

│

├── README.md

├── requirements.txt

├── data/

├── notebooks/

├── src/

├── models/

└── results/

README File Example

Should contain:

Project Title

Example:

Customer Churn Prediction

Description

Explain:

Problem
Dataset
Approach
Results

Installation

Example:

pip install -r requirements.txt

Usage

Example:

python app.py

Jupyter Notebook Portfolio

Jupyter notebooks are useful for:

Data analysis
Visualization
Model experiments

Structure:

1. Import Libraries

2. Load Data

3. Clean Data

4. Analyze Data

5. Build Model

6. Results

Data Science Tools

Programming

Python
SQL

Data Analysis

Pandas
NumPy

Visualization

Matplotlib
Plotly

Machine Learning

Scikit-learn

Deep Learning

TensorFlow
PyTorch

Deployment

Flask
FastAPI
Docker

Building a Professional Resume

Highlight:

Technical Skills

Example:

Python
Machine Learning
SQL
Data Analysis
Deep Learning

Projects

Include:

Project name
Technology used
Results

Example:

Customer Churn Prediction

Built ML model using Python and achieved
high prediction accuracy.

Career Paths After Learning Python

Python Developer

Works on:

Applications
APIs
Automation

Data Analyst

Works on:

Reports
Dashboards
Insights

Data Scientist

Works on:

ML models
Predictions

Machine Learning Engineer

Works on:

AI systems
Model deployment

AI Engineer

Works on:

Deep learning
Generative AI

Final Python Learning Roadmap

Python Basics

↓

Advanced Python

↓

Data Analysis

↓

Machine Learning

↓

Deep Learning

↓

AI

↓

Cloud

↓

Projects

↓

Career

Practice Tasks

Create your GitHub account.
Upload your first Python project.
Build a data analysis project.
Create a machine learning project.
Deploy a Python application.
Write project documentation.

Chapter 65: Advanced Python Interview Preparation

Why Python Interview Preparation?

Learning Python is not only about writing code. Professional developers must understand:

How Python works internally
Problem-solving techniques
Data structures
Algorithms
Object-oriented programming
Real-world application design

Python Interview Preparation Roadmap

Python Basics

      ↓

Advanced Python

      ↓

Data Structures

      ↓

Algorithms

      ↓

System Design

      ↓

Real Projects

      ↓

Interview Success

Section 1: Python Fundamentals Interview Questions

Q1. What is Python?

Answer:

Python is a high-level, interpreted, general-purpose programming language known for:

Simple syntax
Dynamic typing
Large library support
Object-oriented programming

Q2. What are Python Features?

Python provides:

Easy syntax
Cross-platform support
Large standard library
Automatic memory management
Object-oriented programming
Functional programming support

Q3. Difference Between List and Tuple

List	Tuple
Mutable	Immutable
Uses []	Uses ()
Slower	Faster
More memory	Less memory

Example:

List:

numbers=[1,2,3]

Tuple:

numbers=(1,2,3)

Q4. What is Mutable and Immutable?

Mutable

Objects that can change.

Examples:

list
dict
set

Immutable

Objects that cannot change.

Examples:

int
str
tuple

Q5. What is Dynamic Typing?

Python automatically determines variable type.

Example:

x=10

x="Python"

Same variable can store different types.

Section 2: Python Memory Management

How Python Manages Memory?

Python uses:

Private heap
Garbage collector
Reference counting

Reference Counting

Python tracks how many variables refer to an object.

Example:

a=[1,2,3]

b=a

Both point to same object.

Garbage Collection

Automatically removes unused objects.

Example:

del a

Section 3: Advanced Python Concepts

Q6. What are *args and **kwargs?

*args

Accepts multiple positional arguments.

Example:

def add(*numbers):

    return sum(numbers)

**kwargs

Accepts multiple keyword arguments.

Example:

def info(**data):

    print(data)

Q7. What is a Lambda Function?

A small anonymous function.

Example:

square=lambda x:x*x

Q8. What are Decorators?

Decorators modify the behavior of functions.

Example:

@decorator
def hello():
    pass

Uses:

Logging
Authentication
Timing

Q9. What are Generators?

Generators produce values one at a time using:

yield

Example:

def numbers():

    yield 1
    yield 2

Advantages:

Memory efficient
Good for large data

Q10. Difference Between Iterator and Generator

Iterator	Generator
Uses iter()	Uses yield
More code	Less code
Class based	Function based

Section 4: Object-Oriented Programming Questions

Q11. What is OOP?

Object-Oriented Programming organizes programs using objects.

Main concepts:

Class
Object
Inheritance
Polymorphism
Encapsulation
Abstraction

Q12. Class vs Object

Class:

Blueprint

Object:

Instance of class

Example:

class Car:
    pass


car1=Car()

Q13. What is Inheritance?

A class acquiring properties from another class.

Example:

class Animal:

    def sound(self):
        pass


class Dog(Animal):

    pass

Q14. What is Polymorphism?

Same method name with different behavior.

Example:

len("Python")

len([1,2,3])

Q15. What is Encapsulation?

Wrapping data and methods together.

Example:

class Bank:

    def __init__(self):

        self.__balance=0

Section 5: Data Structures Interview Questions

Stack

Works on:

LIFO

Last In First Out

Example:

Stack of plates

Python:

stack=[]

stack.append(10)

stack.pop()

Queue

Works on:

FIFO

First In First Out

Python:

from collections import deque


queue=deque()

queue.append(10)

queue.popleft()

Dictionary

Stores key-value pairs.

Example:

student={
"name":"John",
"age":20
}

Set

Stores unique values.

Example:

numbers={1,2,3}

Section 6: Algorithm Interview Questions

Time Complexity

Measures algorithm efficiency.

Example:

O(1)

O(n)

O(log n)

O(n²)

Searching Algorithms

Linear Search

Checks every element.

Complexity:

O(n)

Binary Search

Works on sorted data.

Complexity:

O(log n)

Sorting Algorithms

Common:

Bubble Sort
Merge Sort
Quick Sort

Python Sorting

numbers.sort()

or:

sorted(numbers)

Section 7: Common Coding Interview Problems

Problem 1: Reverse String

Input:

Python

Output:

nohtyP

Solution:

text="Python"

reverse=text[::-1]

print(reverse)

Problem 2: Find Maximum Number

numbers=[5,10,2,8]


maximum=max(numbers)


print(maximum)

Problem 3: Count Characters

text="python"


count={}


for char in text:

    count[char]=count.get(char,0)+1

Problem 4: Remove Duplicates

numbers=[1,2,2,3]


result=list(set(numbers))

Problem 5: Fibonacci Series

a=0
b=1


for i in range(10):

    print(a)

    a,b=b,a+b

Section 8: Database Interview Questions

SQL Basics

Common commands:

SELECT

INSERT

UPDATE

DELETE

Difference Between SQL and NoSQL

SQL	NoSQL
Tables	Documents
Structured	Flexible
MySQL	MongoDB

Python Database Connection

Example:

import sqlite3


connection=sqlite3.connect(
"data.db"
)

Section 9: Web Development Interview Questions

Flask vs Django

Flask	Django
Lightweight	Full framework
Flexible	Built-in features
Small apps	Large apps

REST API

REST API allows applications to communicate using HTTP.

Methods:

GET
POST
PUT
DELETE

Section 10: Machine Learning Interview Questions

What is Machine Learning?

Machine learning allows computers to learn patterns from data.

Types of ML

Supervised Learning
Unsupervised Learning
Reinforcement Learning

Overfitting

When a model learns training data too well.

Result:

Good training performance
Poor real-world performance

Underfitting

Model is too simple.

Result:

Poor performance everywhere

Train-Test Split

Purpose:

Evaluate model performance.

Example:

train_test_split()

Chapter 66: Python Career Roadmap and Future Learning Path

Introduction

Learning Python opens many career opportunities because Python is used in:

Software development
Data science
Artificial intelligence
Automation
Cyber security
Cloud computing
Web development

A successful Python career requires a combination of:

Programming skills
Problem-solving ability
Projects
Industry knowledge

Complete Python Career Roadmap

Python Basics

      ↓

Advanced Python

      ↓

Choose Career Path

      ↓

Build Projects

      ↓

Create Portfolio

      ↓

Interview Preparation

      ↓

Professional Career

Step 1: Master Python Fundamentals

Learn:

Variables
Data types
Operators
Conditions
Loops
Functions
Modules
File handling
Exception handling

Goal:

Build strong programming foundations.

Step 2: Learn Advanced Python

Topics:

Object-oriented programming
Decorators
Generators
Iterators
Context managers
Multithreading
Multiprocessing
Async programming

Goal:

Write professional Python applications.

Step 3: Choose Your Python Career Path

Python has multiple career directions.

Career Path 1: Python Developer

What They Do

Build:

Applications
APIs
Backend systems
Automation tools

Skills Required

Python:

OOP
Data structures
Algorithms

Frameworks:

Django
Flask
FastAPI

Database:

SQL
PostgreSQL
MySQL

Tools:

Git
Docker

Projects

Beginner:

Calculator app
File manager

Intermediate:

Blog application
REST API

Advanced:

Enterprise backend system

Career Path 2: Data Analyst

What They Do

Convert data into business insights.

Skills Required

Python:

Pandas
NumPy

Visualization:

Matplotlib
Power BI
Tableau

Database:

Statistics:

Probability
Data analysis

Projects

Examples:

Sales dashboard
Customer analysis
Market analysis

Career Path 3: Data Scientist

What They Do

Build prediction models using data.

Skills Required

Python:

Pandas
NumPy
Scikit-learn

Statistics:

Probability
Hypothesis testing

Machine Learning:

Regression
Classification
Clustering

Projects

Examples:

Customer churn prediction
Price prediction
Recommendation system

Career Path 4: Machine Learning Engineer

What They Do

Build and deploy ML systems.

Skills Required

Machine Learning:

Algorithms
Model optimization

Deep Learning:

TensorFlow
PyTorch

Deployment:

APIs
Docker
Cloud

Projects

Examples:

Image recognition system
AI prediction platform
Recommendation engine

Career Path 5: AI Engineer

What They Do

Build intelligent applications.

Skills Required

Deep Learning:

Neural networks
CNN
Transformers

AI:

Natural Language Processing
Computer Vision
Generative AI

Tools:

TensorFlow
PyTorch
AI APIs

Projects

Examples:

AI chatbot
Voice assistant
Image generation application

Career Path 6: Automation Engineer

What They Do

Automate repetitive tasks.

Skills Required

Python:

Scripts
APIs
File handling

Tools:

Selenium
Requests
Automation libraries

Projects

Examples:

Email automation
Report generator
Web automation bot

Career Path 7: Cyber Security Python Developer

What They Do

Create security tools and automation systems.

Skills Required

Python:

Networking
Scripting

Security:

Encryption
Logs
Security concepts

Projects

Examples:

Log analyzer
File integrity checker
Security monitoring tool

Career Path 8: Cloud Python Developer

What They Do

Build cloud-based applications.

Skills Required

Cloud:

AWS
Google Cloud
Azure

Python:

Cloud APIs
Automation

Tools:

Docker
Kubernetes

Projects

Examples:

Cloud file storage system
Serverless application
Cloud automation tool

Chapter 67: Building Real-World Python Projects (Complete Hands-On Guide)

Introduction

Learning Python concepts is important, but building real-world projects is what transforms you into a professional developer.

Projects help you:

Apply programming knowledge
Solve real problems
Build a portfolio
Prepare for jobs
Understand software development workflow

Real-World Project Development Process

Professional developers follow this workflow:

Project Idea

      ↓

Requirement Analysis

      ↓

Design Architecture

      ↓

Write Code

      ↓

Testing

      ↓

Documentation

      ↓

Deployment

      ↓

Maintenance

Step 1: Choosing a Project Idea

A good project should:

Solve a real problem
Teach new skills
Be expandable
Have practical value

Types of Python Projects

Python Projects

        |

--------------------------------

|        |          |           |

Web     Data       AI        Automation

Apps   Projects  Projects    Tools

Project Category 1: Python Beginner Projects

Project 1: Calculator Application

Features

Addition
Subtraction
Multiplication
Division

Skills Learned

Functions
Conditions
User input

Example:

def add(a,b):

    return a+b


print(add(10,5))

Project 2: Password Generator

Features

Generate random passwords
Select password length
Include symbols

Skills:

Random module
String handling

Example:

import random


password=random.randint(
1000,
9999
)

print(password)

Project 3: Expense Tracker

Features

Add expenses
View records
Calculate totals

Skills:

File handling
Data structures

Project Category 2: Automation Projects

Project 4: Automatic File Organizer

Problem

Downloads folder becomes messy.

Solution:

Automatically organize files.

Features:

Detect file types
Create folders
Move files

Skills:

os module
shutil module

Project 5: Email Automation System

Features

Send automatic emails
Attach reports
Schedule messages

Skills:

SMTP
Scheduling

Project 6: Web Scraper

Purpose

Collect information automatically from websites.

Applications:

Price tracking
News collection
Data gathering

Libraries:

Requests
BeautifulSoup

Project Category 3: Web Development Projects

Project 7: Blog Website

Features

User registration
Create posts
Comments
Admin panel

Technologies:

Django
SQLite/PostgreSQL
HTML/CSS

Project 8: REST API Application

Features

Create API endpoints
Store data
Return JSON responses

Framework:

FastAPI
Flask

Example API response:

{
"name":"Python",
"level":"Advanced"
}

Project 9: E-Commerce Website

Features

Product listing
Shopping cart
User accounts
Payments

Skills:

Backend development
Database design
Authentication

Project Category 4: Data Science Projects

Project 10: Sales Analysis Dashboard

Goal

Analyze company sales data.

Features:

Revenue analysis
Monthly trends
Product performance

Libraries:

Pandas
Matplotlib

Project 11: Customer Churn Prediction

Goal

Predict customers who may leave a service.

Skills:

Data cleaning
Machine learning
Model evaluation

Algorithms:

Logistic Regression
Random Forest

Project 12: Stock Market Analysis

Features

Analyze stock data
Visualize trends
Generate reports

Skills:

APIs
Data visualization

Project Category 5: Artificial Intelligence Projects

Project 13: Chatbot

Features

Answer user questions
Process text
Provide responses

Skills:

NLP
Machine learning

Project 14: Image Recognition System

Features

Detect objects
Classify images

Technologies:

OpenCV
TensorFlow

Project 15: Recommendation System

Examples:

Movie recommendations
Product suggestions

Skills:

Machine learning
Data processing

Project Category 6: Cyber Security Projects

Project 16: Password Security Checker

Features:

Check password strength
Detect weak passwords

Skills:

Security concepts
Regular expressions

Project 17: Log Analyzer

Features:

Read server logs
Detect suspicious activity

Skills:

File processing
Pattern detection

Project 18: File Integrity Monitor

Purpose:

Detect unauthorized file changes.

Skills:

Hashing
Security automation

Project Category 7: Cloud Projects

Project 19: Cloud File Storage System

Features:

Upload files
Download files
Manage storage

Technologies:

Python
AWS S3

Project 20: Server Monitoring System

Monitor:

CPU usage
Memory
Disk space

Skills:

Cloud monitoring
Automation

Professional Project Structure

A good Python project should look like:

project/

│

├── README.md

├── requirements.txt

├── main.py

├── config.py

├── database/

├── tests/

├── modules/

└── documentation/

Writing Clean Code

Follow:

Meaningful Names

Bad:

x=10

Good:

user_age=10

Use Functions

Avoid:

100 lines of code

Use:

small reusable functions

Add Comments

Explain complex logic.

Version Control with Git

Git helps track code changes.

Basic commands:

git init

git add .

git commit -m "first version"

Testing Projects

Testing ensures software works correctly.

Python testing tools:

unittest
pytest

Example:

def test_add():

    assert add(2,3)==5

Documentation

A professional project includes:

README

Contains:

Project description
Installation
Usage

Requirements File

Contains libraries:

flask

pandas

numpy

Deploying Python Projects

Deployment options:

Web Apps

Cloud servers
Hosting platforms

APIs

Docker
Cloud services

AI Models

FastAPI
Cloud deployment

How to Make Projects Stand Out

Add:

✅ Good UI
✅ Documentation
✅ Tests
✅ Error handling
✅ Database support
✅ Deployment
✅ Real-world problem solving

Portfolio Project Roadmap

Beginner Portfolio

Create:

Calculator
Expense tracker
File organizer

Intermediate Portfolio

Create:

Website
API
Automation system

Advanced Portfolio

Create:

AI application
Cloud application
Complete software system

Final Advice for Project Building

Remember:

Start small
Improve gradually
Write clean code
Document everything
Publish projects
Learn from mistakes

Chapter 68: Python Software Development Best Practices

Introduction

Writing Python code that works is only the beginning. Professional developers write code that is:

Easy to read
Easy to maintain
Secure
Testable
Efficient
Scalable

Good software development practices help teams build reliable applications.

Professional Python Development Workflow

Requirement

     ↓

Design

     ↓

Development

     ↓

Testing

     ↓

Code Review

     ↓

Deployment

     ↓

Maintenance

1. Follow PEP 8 Style Guide

PEP 8 is the official Python coding style guide.

It improves:

Readability
Consistency
Team collaboration

Naming Conventions

Variables

Use lowercase with underscores:

user_name = "Alex"

total_price = 500

Functions

Use descriptive names:

Good:

calculate_total()

Bad:

ct()

Classes

Use PascalCase:

class UserAccount:
    pass

2. Write Clean Code

Clean code should be:

Simple
Understandable
Organized

Avoid Complex Code

Bad:

if x==1:
    if y==2:
        print("Yes")

Better:

if x==1 and y==2:
    print("Yes")

3. Use Functions Properly

Functions should:

Do one task
Have clear names
Be reusable

Example:

def calculate_tax(price):

    return price * 0.18

4. Avoid Repeated Code

Bad:

print("Hello")
print("Hello")
print("Hello")

Better:

def greet():

    print("Hello")


for i in range(3):

    greet()

5. Use Comments and Documentation

Comments explain why code exists.

Example:

# Calculate final price after discount

final_price = price - discount

Docstrings

Used to describe functions.

Example:

def add(a,b):
    """
    Adds two numbers.
    """

    return a+b

6. Error Handling

Programs should handle unexpected situations.

Using try-except

try:

    number=int(input())

except ValueError:

    print("Invalid input")

Handling Multiple Errors

try:

    file=open("data.txt")

except FileNotFoundError:

    print("File missing")

except PermissionError:

    print("Access denied")

7. Logging Instead of Print

Professional applications use logging.

Example:

import logging


logging.info(
"Application started"
)

Logging helps track:

Errors
Warnings
System activity

8. Project Organization

Large projects should be structured.

Example:

application/

│

├── main.py

├── database/

├── models/

├── services/

├── tests/

└── config.py

9. Use Virtual Environments

Virtual environments isolate project dependencies.

Create:

python -m venv env

Activate:

Windows:

env\Scripts\activate

Linux:

source env/bin/activate

10. Manage Dependencies

Use:

requirements.txt

Example:

django

pandas

requests

Install:

pip install -r requirements.txt

11. Version Control with Git

Git tracks code changes.

Important commands:

Initialize:

git init

Add files:

git add .

Commit:

git commit -m "update code"

Git Best Practices

Use:

Small commits
Clear messages
Branches

Example:

main

 |

feature-login

feature-payment

12. Testing in Python

Testing ensures code reliability.

Types:

Unit testing
Integration testing
System testing

Unit Testing

Tests individual functions.

Example:

def add(a,b):

    return a+b

Test:

assert add(2,3)==5

Testing Libraries

unittest

Built into Python.

pytest

Popular testing framework.

Install:

pip install pytest

13. Debugging Techniques

Debugging finds and fixes errors.

Using print()

Simple debugging:

print(variable)

Using Debugger

Tools:

VS Code debugger
PyCharm debugger

Reading Error Messages

Understand:

Error type
Line number
Cause

Common Python Errors

Syntax Error

Wrong code format.

Example:

print("Hello"

Type Error

Wrong data type operation.

Example:

"5"+10

Index Error

Wrong list position.

Example:

numbers[10]

14. Performance Optimization

Efficient code runs faster.

Use Built-in Functions

Better:

sum(numbers)

Instead of:

total=0

for n in numbers:

    total+=n

Use Generators for Large Data

Instead of:

numbers=[x for x in range(1000000)]

Use:

numbers=(x for x in range(1000000))

Optimize Database Queries

Avoid:

Unnecessary queries
Loading unused data

15. Security Best Practices

Professional applications must be secure.

Never Store Passwords Directly

Bad:

password="123456"

Use:

Hashing
Environment variables

Validate User Input

Never trust user data.

Example:

if age.isdigit():

    print(age)

Protect API Keys

Use:

import os


key=os.getenv(
"API_KEY"
)

16. Design Patterns in Python

Design patterns are reusable solutions to common problems.

Singleton Pattern

Ensures only one object exists.

Examples:

Database connection
Configuration manager

Factory Pattern

Creates objects dynamically.

Example:

class Factory:

    def create():

        return Object()

Observer Pattern

Used for event systems.

Examples:

Notifications
Updates

17. Object-Oriented Best Practices

Follow:

Single Responsibility Principle

A class should have one responsibility.

Open/Closed Principle

Code should be open for extension but closed for modification.

Avoid Large Classes

Split responsibilities.

18. Code Review Practices

Code reviews improve quality.

Check:

Readability
Security
Performance
Testing

19. Continuous Integration (CI)

CI automatically checks code changes.

Tools:

GitHub Actions
Jenkins

Workflow:

Code Push

    ↓

Run Tests

    ↓

Build

    ↓

Deploy

20. Production-Level Python Checklist

Before releasing software:

✅ Clean code
✅ Error handling
✅ Tests written
✅ Security checked
✅ Documentation completed
✅ Dependencies managed
✅ Performance tested
✅ Deployment prepared

Practice Projects

Refactor an old Python project.
Add tests to an application.
Create a logging system.
Build a clean API project.
Deploy a production-ready application.

Chapter 69: Python System Design and Architecture

Introduction

System Design is the process of planning how a software system will be built, organized, and scaled.

A professional Python developer should understand:

Application architecture
Database design
API design
Scalability
Performance
Security

What is Software Architecture?

Software architecture defines the structure of an application.

It describes:

Components
Communication between components
Data flow
Technology choices

Basic Application Architecture

User

 ↓

Frontend

 ↓

API Layer

 ↓

Business Logic

 ↓

Database

System Design Goals

A good system should be:

Scalable

Handle increasing users.

Reliable

Continue working without failures.

Secure

Protect data and users.

Maintainable

Easy to update and improve.

Fast

Provide quick responses.

Types of Software Architecture

1. Monolithic Architecture

All components exist in one application.

Example:

Application

 ├── User Module

 ├── Payment Module

 └── Product Module

Advantages

Simple development
Easy deployment

Disadvantages

Difficult to scale
Large codebase

2. Microservices Architecture

Application is divided into small independent services.

Example:

User Service

Payment Service

Product Service

Notification Service

Advantages

Easy scaling
Independent deployment

Disadvantages

More complexity
Requires communication between services

Python Application Layers

A common Python architecture:

Presentation Layer

        ↓

API Layer

        ↓

Service Layer

        ↓

Database Layer

Layer 1: API Layer

Handles:

HTTP requests
Responses
Authentication

Frameworks:

FastAPI
Flask
Django REST Framework

Example:

@app.get("/users")
def users():

    return {
        "message":"Users list"
    }

Layer 2: Business Logic Layer

Contains application rules.

Example:

def calculate_discount(price):

    if price > 1000:

        return price*0.9

Layer 3: Database Layer

Handles:

Saving data
Retrieving data

Technologies:

PostgreSQL
MySQL
MongoDB

Database Design Basics

Database Tables

Example:

Users table:

id	name	email
1	John	john@email.com

Relationships

One-to-One

One record connects to one record.

Example:

User → Profile

One-to-Many

One record connects to many records.

Example:

Customer → Orders

Many-to-Many

Multiple records connect together.

Example:

Students ↔ Courses

Object Relational Mapping (ORM)

ORM allows Python objects to interact with databases.

Examples:

SQLAlchemy
Django ORM

Example:

class User:

    name="John"

Stored as database record.

API Design

What is an API?

API allows applications to communicate.

Example:

Mobile App

      ↓

API

      ↓

Database

REST API Principles

REST uses HTTP methods.

GET

Retrieve data.

Example:

GET /users

POST

Create data.

Example:

POST /users

PUT

Update data.

Example:

PUT /users/1

DELETE

Remove data.

Example:

DELETE /users/1

API Response Format

Usually JSON:

{
"id":1,
"name":"Python"
}

Authentication Design

Common methods:

Session Authentication

Used in websites.

Token Authentication

Used in APIs.

Example:

User

 ↓

Login

 ↓

Token

 ↓

API Access

JWT Authentication

JWT contains:

User information
Expiration time
Signature

Python libraries:

PyJWT

Caching

What is Caching?

Caching stores frequently used data temporarily.

Benefits:

Faster response
Less database load

Example:

Without cache:

User

 ↓

Database

 ↓

Response

With cache:

User

 ↓

Cache

 ↓

Response

Popular Cache Systems

Redis
Memcached

Load Balancing

What is Load Balancing?

Distributes traffic across multiple servers.

Example:

          Users

            ↓

      Load Balancer

       /     |     \

 Server1 Server2 Server3

Benefits:

High availability
Better performance

Message Queues

Used for background tasks.

Examples:

Sending emails
Processing files

Popular tools:

RabbitMQ
Kafka
Celery

Background Tasks in Python

Example:

def send_email():

    print("Email sent")

Can run separately from main application.

File Storage Design

Large files should not always be stored in databases.

Use:

Cloud storage
Object storage

Examples:

AWS S3
Google Cloud Storage

Scaling Python Applications

Vertical Scaling

Increase server power.

Example:

More CPU

More RAM

Horizontal Scaling

Add more servers.

Example:

Server 1

Server 2

Server 3

Python Performance Optimization

Techniques:

Caching
Database optimization
Async programming
Load balancing
Efficient algorithms

Async Programming

Useful for:

APIs
Network tasks
Multiple requests

Example:

async def fetch_data():

    return "data"

Containerization with Docker

Docker packages applications.

Structure:

Application

+

Dependencies

+

Environment

=

Container

Kubernetes Basics

Kubernetes manages containers.

Used for:

Scaling
Deployment
Monitoring

Monitoring Systems

Production applications need monitoring.

Monitor:

CPU
Memory
Errors
Response time

Tools:

Prometheus
Grafana

Logging Architecture

Good systems use centralized logging.

Example:

Application Logs

        ↓

Log System

        ↓

Monitoring Dashboard

Security Architecture

Important areas:

Authentication

Verify users.

Authorization

Control permissions.

Encryption

Protect data.

Input Validation

Prevent attacks.

Real-World System Design Example

Online Shopping System

Architecture:

User

 ↓

Frontend

 ↓

API Gateway

 ↓

Services

 ├── User Service

 ├── Product Service

 ├── Order Service

 └── Payment Service

 ↓

Database

Python Technologies Used

Backend:

Django
FastAPI

Database:

PostgreSQL

Cache:

Redis

Queue:

Celery

Deployment:

Docker
Cloud

System Design Interview Questions

Common questions:

Design a URL shortener.
Design an online store.
Design a chat application.
Design a notification system.
Design a file storage system.

Practice Projects

Beginner

REST API application
Database management system

Intermediate

Blog platform
E-commerce backend

Advanced

Microservice application
Cloud-scale system
Real-time chat platform

Chapter 70: Python DevOps and Deployment Engineering

Introduction

DevOps combines software development and IT operations to build, test, deploy, and maintain applications faster and more reliably.

Python plays an important role in DevOps because it is widely used for:

Automation scripts
Deployment tools
Cloud management
Infrastructure automation
Monitoring systems

What is DevOps?

DevOps is a culture and practice that connects:

Development Team

        +

Operations Team

        ↓

Reliable Software Delivery

DevOps Lifecycle

Plan

 ↓

Code

 ↓

Build

 ↓

Test

 ↓

Release

 ↓

Deploy

 ↓

Monitor

 ↓

Improve

Why Python in DevOps?

Python helps with:

Automating repetitive tasks
Managing servers
Creating deployment scripts
Working with cloud APIs
Monitoring applications

Python DevOps Tools

Common tools:

Automation

Ansible
Fabric

Containers

Docker
Kubernetes

CI/CD

Jenkins
GitHub Actions

Cloud

AWS
Azure
Google Cloud

Section 1: Environment Management

Development Environment

A developer environment contains:

Python version
Libraries
Configuration
Tools

Virtual Environments

Create:

python -m venv env

Activate:

Windows:

env\Scripts\activate

Linux:

source env/bin/activate

Dependency Management

Store packages:

requirements.txt

Example:

django

requests

numpy

Install:

pip install -r requirements.txt

Section 2: Version Control with Git

Git tracks changes in software projects.

Basic Git Workflow

Write Code

    ↓

Git Add

    ↓

Commit

    ↓

Push

    ↓

Repository

Important Commands

Initialize:

git init

Add files:

git add .

Commit:

git commit -m "first commit"

Push:

git push

Branching Strategy

Branches allow parallel development.

Example:

main

 |

feature-login

feature-payment

Section 3: Continuous Integration (CI)

What is CI?

Continuous Integration automatically tests code when changes are made.

Workflow:

Developer Pushes Code

          ↓

Automatic Tests

          ↓

Build Verification

          ↓

Report Result

CI Benefits

Finds bugs early
Improves code quality
Saves time

Popular CI Tools

GitHub Actions
Jenkins
GitLab CI

GitHub Actions Example

File:

.github/workflows/test.yml

Example:

name: Python Test

on:
  push:

jobs:
  test:

    runs-on: ubuntu-latest

    steps:

    - uses: actions/checkout@v3

    - name: Run tests
      run: pytest

Section 4: Continuous Deployment (CD)

What is CD?

Continuous Deployment automatically releases tested software.

Workflow:

Code

 ↓

Test

 ↓

Build

 ↓

Deploy

 ↓

Production

Deployment Strategies

1. Direct Deployment

Simple deployment.

2. Blue-Green Deployment

Two environments:

Blue = Current Version

Green = New Version

3. Rolling Deployment

Gradually updates servers.

Section 5: Docker with Python

What is Docker?

Docker packages applications into containers.

A container includes:

Application code
Python
Libraries
Configuration

Docker Architecture

Application

      +

Dependencies

      +

Runtime

      ↓

Container

Dockerfile Example

FROM python:3.12

WORKDIR /app

COPY requirements.txt .

RUN pip install -r requirements.txt

COPY . .

CMD ["python","app.py"]

Docker Commands

Build image:

docker build -t myapp .

Run container:

docker run myapp

List containers:

docker ps

Docker Compose

Used for multiple services.

Example:

Application

 +

Database

 +

Cache

Section 6: Kubernetes Basics

What is Kubernetes?

Kubernetes manages containers automatically.

It handles:

Scaling
Deployment
Recovery

Kubernetes Architecture

Master Node

      ↓

Worker Nodes

      ↓

Containers

Kubernetes Features

Automatic scaling
Load balancing
Self-healing
Service discovery

Kubernetes Objects

Pod

Smallest deployment unit.

Service

Provides network access.

Deployment

Manages application versions.

Section 7: Cloud Deployment

Python applications can be deployed on:

AWS

Services:

EC2
Lambda
Elastic Beanstalk

Google Cloud

Services:

Cloud Run
App Engine

Azure

Services:

Azure App Service
Functions

Deploying Flask Application

Example structure:

app/

├── app.py

├── requirements.txt

└── Dockerfile

Section 8: Infrastructure Automation

What is Infrastructure as Code (IaC)?

Managing infrastructure using code.

Instead of:

Manual Server Setup

Use:

Automated Configuration

Tools

Terraform
Ansible

Ansible with Python

Ansible automates:

Server setup
Application deployment
Configuration management

Section 9: Monitoring Applications

Production systems need monitoring.

Track:

CPU usage
Memory
Errors
Response time

Monitoring Tools

Prometheus

Collects metrics.

Grafana

Creates dashboards.

ELK Stack

Handles logs:

Elasticsearch
Logstash
Kibana

Python Monitoring Example

import psutil


cpu=psutil.cpu_percent()


print(cpu)

Section 10: Security in DevOps

DevSecOps

Security integrated into DevOps.

Security Practices

Protect Secrets

Use:

Environment variables
Secret managers

Scan Dependencies

Check libraries for vulnerabilities.

Secure Containers

Use:

Updated images
Minimal packages

Section 11: Production Deployment Checklist

Before deployment:

✅ Code tested
✅ Dependencies fixed
✅ Environment configured
✅ Security checked
✅ Database prepared
✅ Monitoring enabled
✅ Backup configured

Real-World DevOps Projects

Beginner Projects

Deploy Flask application
Create CI pipeline
Dockerize Python app

Intermediate Projects

Kubernetes deployment
Automated cloud deployment
Monitoring dashboard

Advanced Projects

Complete CI/CD platform
Cloud infrastructure automation
Microservices deployment system

Chapter 71: Python Blockchain and Web3 Development

Introduction

Blockchain and Web3 are emerging technologies that focus on decentralized applications, digital ownership, and secure data systems.

Python is used in blockchain development for:

Blockchain research
Automation tools
Data analysis
Security applications
Backend services
Web3 integrations

What is Blockchain?

A blockchain is a distributed digital ledger that stores information in connected blocks.

Each block contains:

Data
Timestamp
Previous block reference
Hash value

Blockchain Structure

Block 1

   ↓

Block 2

   ↓

Block 3

   ↓

Block 4

Each block is connected to the previous block.

Features of Blockchain

Decentralization

No single organization controls the entire network.

Transparency

Transactions can be verified by network participants.

Security

Cryptographic techniques protect data.

Immutability

Once recorded, data is difficult to change.

How Blockchain Works

Transaction Created

        ↓

Transaction Verification

        ↓

Block Creation

        ↓

Network Validation

        ↓

Block Added

        ↓

Ledger Updated

What is Web3?

Web3 represents a decentralized version of the internet.

Traditional Web:

User

 ↓

Central Server

 ↓

Application

Web3:

User

 ↓

Blockchain Network

 ↓

Decentralized Application

Web1 vs Web2 vs Web3

Version	Description
Web1	Read-only websites
Web2	Interactive apps controlled by companies
Web3	Decentralized applications

Blockchain Terminology

Node

A computer participating in a blockchain network.

Wallet

A digital tool used to manage blockchain accounts.

Address

A public identifier for receiving assets.

Transaction

A record of an action on blockchain.

Smart Contract

A program stored on blockchain that executes automatically.

Cryptography Basics

Blockchain uses cryptography for security.

Important concepts:

Hashing
Digital signatures
Encryption

Hash Function

A hash converts data into a fixed-length value.

Example:

Input

"Python"

↓

Hash

"8f23ab..."

Python Hash Example

import hashlib


data="Python"


hash_value=hashlib.sha256(
data.encode()
)


print(hash_value.hexdigest())

Building a Simple Blockchain in Python

Creating a Block

import hashlib
import datetime


class Block:

    def __init__(self,data):

        self.data=data

        self.time=datetime.datetime.now()

        self.hash=self.calculate_hash()


    def calculate_hash(self):

        return hashlib.sha256(
            str(self.data).encode()
        ).hexdigest()

Creating Blockchain

class Blockchain:

    def __init__(self):

        self.chain=[]


    def add_block(self,block):

        self.chain.append(block)

Cryptocurrency Basics

A cryptocurrency is a digital asset secured by blockchain technology.

Examples:

Bitcoin
Ethereum

Bitcoin Concepts

Bitcoin introduced:

Digital currency
Mining
Proof of Work
Decentralized payments

Ethereum Concepts

Ethereum introduced:

Smart contracts
Decentralized applications
Token systems

Smart Contracts

A smart contract is a program that runs automatically when conditions are met.

Example:

IF payment received

THEN transfer ownership

Smart Contract Languages

Common languages:

Solidity
Vyper

Python and Ethereum

Python can interact with Ethereum networks using:

Web3.py

Installing Web3.py

pip install web3

Connecting to Ethereum

Example:

from web3 import Web3


web3 = Web3(
Web3.HTTPProvider(
"node_url"
)
)


print(
web3.is_connected()
)

Reading Blockchain Data

Example:

latest_block = web3.eth.block_number

print(latest_block)

Smart Contract Interaction

Python can:

Read contract data
Send transactions
Monitor events

Decentralized Applications (DApps)

A DApp contains:

Frontend

    ↓

Web3 Connection

    ↓

Smart Contract

    ↓

Blockchain

Types of Blockchain Applications

1. Financial Applications

Examples:

Digital payments
Decentralized finance (DeFi)

2. Supply Chain Systems

Used for:

Product tracking
Transparency

3. Digital Identity

Used for:

Identity verification
Ownership records

4. Gaming Applications

Examples:

Digital assets
Virtual ownership

5. NFT Applications

NFTs represent unique digital ownership.

Examples:

Digital artwork
Collectibles

Blockchain Data Analysis with Python

Python can analyze:

Transactions
Wallet activity
Network statistics

Libraries:

Pandas
NumPy
Matplotlib

Example Blockchain Data Analysis

import pandas as pd


data=pd.read_csv(
"transactions.csv"
)


print(
data.head()
)

Blockchain Security with Python

Python is used for:

Security testing
Cryptography experiments
Network analysis

Common Security Concepts

Private Keys

Secret keys used to authorize transactions.

Public Keys

Used to identify accounts.

Digital Signatures

Verify transaction ownership.

Web3 Development Tools

Python Libraries

Web3.py
eth-account
Brownie

Blockchain Platforms

Ethereum
Polygon
Solana
Hyperledger

Blockchain Development Roadmap

Python Basics

      ↓

Cryptography

      ↓

Blockchain Concepts

      ↓

Ethereum Basics

      ↓

Smart Contracts

      ↓

Web3.py

      ↓

DApp Development

      ↓

Blockchain Projects

Blockchain Projects Using Python

Beginner Projects

Create a simple blockchain
Build a hash generator
Analyze blockchain data

Intermediate Projects

Crypto price tracker
Wallet information tool
Blockchain explorer

Advanced Projects

DApp backend
Smart contract interaction system
Decentralized application platform

Career Opportunities

Blockchain skills can lead to:

Blockchain Developer

Build blockchain applications.

Web3 Developer

Create decentralized apps.

Smart Contract Developer

Write blockchain programs.

Blockchain Data Analyst

Analyze blockchain activity.

Chapter 72: Python Cloud Computing and Serverless Development

Introduction

Cloud computing allows developers to run applications, store data, and manage services using internet-based infrastructure instead of local machines.

Python is one of the most popular languages for cloud development because it is used for:

Cloud automation
Backend applications
Data processing
AI services
Serverless functions
Infrastructure management

What is Cloud Computing?

Cloud computing provides computing resources through the internet.

Resources include:

Servers
Storage
Databases
Networking
Software services

Traditional Computing vs Cloud Computing

Traditional Server

Company Owns Server

        ↓

Maintains Hardware

        ↓

Runs Applications

Cloud Computing

Cloud Provider

        ↓

Virtual Resources

        ↓

Application Deployment

Benefits of Cloud Computing

Scalability

Increase resources when demand grows.

Cost Efficiency

Pay only for used resources.

Availability

Applications can run globally.

Flexibility

Deploy applications quickly.

Cloud Service Models

Cloud Services

      |

-------------------------

IaaS     PaaS      SaaS

1. Infrastructure as a Service (IaaS)

Provides:

Virtual machines
Storage
Networks

Examples:

AWS EC2
Google Compute Engine
Azure Virtual Machines

2. Platform as a Service (PaaS)

Provides application deployment platforms.

Examples:

AWS Elastic Beanstalk
Google App Engine
Azure App Service

3. Software as a Service (SaaS)

Ready-to-use applications.

Examples:

Email services
Online tools

Major Cloud Providers

Amazon Web Services (AWS)

Most widely used cloud platform.

Services:

EC2
S3
Lambda
RDS

Microsoft Azure

Services:

Virtual Machines
Functions
Databases

Google Cloud Platform (GCP)

Services:

Compute Engine
Cloud Run
BigQuery

Python Cloud Development

Python is used for:

Cloud APIs
Automation
Serverless applications
Data processing

Cloud SDKs for Python

Examples:

AWS

pip install boto3

Google Cloud

pip install google-cloud

Azure

pip install azure

Working with AWS Using Python

Boto3 Library

Boto3 allows Python applications to communicate with AWS.

Install:

pip install boto3

Connecting to AWS

import boto3


s3=boto3.client(
"s3"
)


print(
s3.list_buckets()
)

Cloud Storage with Python

Common storage services:

AWS S3
Google Cloud Storage
Azure Blob Storage

Upload File Example

import boto3


s3=boto3.client(
"s3"
)


s3.upload_file(
"file.txt",
"bucket-name",
"file.txt"
)

Database Services in Cloud

Cloud databases include:

Amazon RDS
Cloud SQL
Azure SQL

Python Database Connection

Example:

import sqlite3


connection=sqlite3.connect(
"database.db"
)

Serverless Computing

What is Serverless?

Serverless allows developers to run code without managing servers.

The cloud provider manages:

Servers
Scaling
Maintenance

Serverless Architecture

User Request

      ↓

Cloud Function

      ↓

Execute Python Code

      ↓

Return Result

Benefits of Serverless

Automatic scaling
Lower cost
Easy deployment
No server management

Popular Serverless Platforms

AWS Lambda

Runs functions on demand.

Google Cloud Functions

Executes cloud-based code.

Azure Functions

Runs event-driven applications.

AWS Lambda with Python

Example:

def lambda_handler(event,context):

    return {

        "statusCode":200,

        "body":"Hello Python Cloud"

    }

Lambda Events

Functions can trigger from:

HTTP requests
File uploads
Database changes
Scheduled tasks

API Gateway + Lambda

Architecture:

User

 ↓

API Gateway

 ↓

Lambda Function

 ↓

Database

Cloud Deployment Workflow

Write Code

      ↓

Test Application

      ↓

Create Cloud Resources

      ↓

Deploy

      ↓

Monitor

      ↓

Scale

Deploying Python Web Applications

Common platforms:

AWS Elastic Beanstalk
Google App Engine
Azure App Service

Example Project Structure

application/

│

├── app.py

├── requirements.txt

├── Dockerfile

└── config.py

Cloud Security Basics

Important practices:

Authentication

Verify users.

Authorization

Control access.

Encryption

Protect information.

Secret Management

Never store passwords in code.

Example:

import os


password=os.getenv(
"DATABASE_PASSWORD"
)

Infrastructure as Code (IaC)

Infrastructure can be created using code.

Tools:

Terraform
CloudFormation

Example:

Instead of manually creating servers:

Create Server

Install Software

Configure Network

Use:

Infrastructure Code

        ↓

Automatic Setup

Containers in Cloud

Docker containers are widely used in cloud deployment.

Architecture:

Python App

      +

Docker Container

      +

Cloud Platform

Kubernetes in Cloud

Kubernetes manages container applications.

Features:

Scaling
Deployment
Recovery
Load balancing

Cloud Monitoring

Monitor:

Application health
Errors
Performance
Resource usage

Tools:

CloudWatch
Azure Monitor
Google Cloud Monitoring

Real-World Cloud Python Projects

Project 1: Cloud File Storage System

Features:

Upload files
Download files
User authentication

Technologies:

Python
AWS S3
Flask/FastAPI

Project 2: Serverless API

Features:

API endpoints
Database connection
Automatic scaling

Technologies:

Python Lambda
API Gateway

Project 3: Cloud Data Pipeline

Features:

Collect data
Process data
Store results

Technologies:

Python
Cloud storage
Cloud databases

Project 4: AI Cloud Application

Features:

AI model deployment
Prediction API
Scalable inference

Technologies:

Python
Machine Learning
Cloud services

Cloud Career Paths

Cloud Python Developer

Works on:

Cloud applications
APIs
Automation

Cloud Engineer

Works on:

Infrastructure
Deployment
Monitoring

DevOps Engineer

Works on:

CI/CD
Containers
Automation

Cloud AI Engineer

Works on:

AI deployment
Machine learning systems

Cloud Learning Roadmap

Python

 ↓

Linux Basics

 ↓

Networking

 ↓

Cloud Fundamentals

 ↓

AWS/Azure/GCP

 ↓

Docker

 ↓

Kubernetes

 ↓

Serverless

 ↓

Cloud Projects

Chapter 73: Python Cyber Security and Ethical Hacking

Introduction

Cyber security focuses on protecting computers, networks, applications, and data from unauthorized access and attacks.

Python is widely used in cyber security because it helps professionals:

Automate security tasks
Analyze data
Build security tools
Test applications
Monitor systems
Perform defensive security operations

What is Cyber Security?

Cyber security is the practice of protecting:

Hardware
Software
Networks
Data
Users

from security threats.

Cyber Security Goals

The main goals are known as the CIA Triad:

        Confidentiality

              |

Integrity -------- Availability

1. Confidentiality

Ensures information is accessible only to authorized users.

Examples:

Encryption
Access control

2. Integrity

Ensures data is not modified incorrectly.

Examples:

Hashing
Digital signatures

3. Availability

Ensures systems remain accessible.

Examples:

Backups
Monitoring

Types of Cyber Security

Network Security

Protects:

Networks
Routers
Communication systems

Application Security

Protects:

Websites
APIs
Software

Data Security

Protects:

Personal data
Business information

Cloud Security

Protects:

Cloud applications
Cloud infrastructure

Ethical Hacking

What is Ethical Hacking?

Ethical hacking is authorized security testing performed to find and fix weaknesses.

Ethical hackers:

Have permission
Follow rules
Report vulnerabilities responsibly

Ethical Hacking Process

Planning

   ↓

Information Gathering

   ↓

Security Testing

   ↓

Analysis

   ↓

Report

   ↓

Fix Issues

Python in Cyber Security

Python is used for:

Security automation
Log analysis
Network monitoring
Malware analysis
Security testing tools

Python Security Libraries

Requests

Used for HTTP communication.

Install:

pip install requests

Example:

import requests


response=requests.get(
"https://example.com"
)


print(response.status_code)

Scapy

Used for network packet analysis.

Install:

pip install scapy

Used for:

Network research
Packet analysis
Security testing

Cryptography Library

Used for:

Encryption
Secure communication

Install:

pip install cryptography

Hashing with Python

Hashing converts data into a fixed value.

Common algorithms:

SHA-256
SHA-512

Example:

import hashlib


message="Python"


hash_value=hashlib.sha256(
message.encode()
)


print(hash_value.hexdigest())

Password Security Concepts

Good password systems use:

Strong hashing
Salt values
Secure storage

Encryption Basics

Encryption converts readable data into protected data.

Original Data

      ↓

Encryption

      ↓

Encrypted Data

Symmetric Encryption

Same key used for:

Encryption
Decryption

Example:

Asymmetric Encryption

Uses:

Public key
Private key

Example:

Network Security Basics

IP Address

Identifies a device on a network.

Port

Identifies a service running on a device.

Examples:

HTTP  → 80

HTTPS → 443

SSH   → 22

Python Networking

Python provides:

socket library
networking automation

Example:

import socket


hostname=socket.gethostname()


print(hostname)

Log Analysis with Python

Security teams analyze logs to find:

Suspicious activity
Errors
Unauthorized access

Example:

with open(
"server.log"
) as file:

    logs=file.readlines()


print(logs)

Regular Expressions in Security

Regex helps search patterns.

Used for:

Log analysis
Data validation

Example:

import re


pattern=r"\d+"


result=re.findall(
pattern,
"Error 404"
)

print(result)

Security Automation Projects

Project 1: Log Analyzer

Features

Read logs
Find errors
Generate reports

Skills:

File handling
Regex
Data analysis

Project 2: Password Strength Checker

Features

Checks:

Length
Characters
Complexity

Skills:

String processing
Validation

Project 3: Security Report Generator

Features:

Collect system information
Generate reports

Skills:

Automation
File generation

Project 4: Network Monitoring Tool

Features:

Monitor connections
Analyze network data

Skills:

Networking
Python automation

Web Application Security Basics

Common security issues:

SQL Injection

Problem:

Unsafe database queries allow unwanted input.

Prevention:

Parameterized queries
ORM usage

Cross-Site Scripting (XSS)

Problem:

Unsafe user input displayed on websites.

Prevention:

Input validation
Output encoding

Authentication Security

Best practices:

Strong passwords
Multi-factor authentication
Secure sessions

Security Testing Tools

Common tools:

Wireshark
Nmap
Burp Suite
OWASP ZAP

OWASP Top Security Risks

Common application risks:

Broken access control
Cryptographic failures
Injection attacks
Security misconfiguration
Authentication problems

Defensive Security with Python

Python can help with:

Monitoring

Track system activity.

Automation

Automate security checks.

Reporting

Generate security reports.

Cyber Security Career Paths

Security Analyst

Works on:

Monitoring
Incident response
Threat detection

Penetration Tester

Works on:

Authorized security testing
Vulnerability assessment

Security Engineer

Works on:

Security systems
Infrastructure protection

Cloud Security Engineer

Works on:

Cloud security
Identity management

Cyber Security Learning Roadmap

Python

 ↓

Networking Basics

 ↓

Linux

 ↓

Security Concepts

 ↓

Cryptography

 ↓

Security Tools

 ↓

Automation

 ↓

Security Projects

Practice Projects

Beginner:

Password checker
File hash generator
Log analyzer

Intermediate:

Security monitoring dashboard
Network analysis tool
Automated security reports

Advanced:

Threat analysis system

Security automation platform

Cloud security monitoring system

Chapter 74: Python Internet of Things (IoT) Development

Introduction

Internet of Things (IoT) connects physical devices to the internet so they can collect data, communicate, and perform automated actions.

Python is widely used in IoT because it is:

Easy to learn
Powerful for automation
Supported on many hardware platforms
Excellent for data processing

What is IoT?

IoT is a network of connected devices that can:

Sense information
Process data
Communicate
Perform actions automatically

Examples:

Smart homes
Smart watches
Industrial sensors
Healthcare devices
Smart agriculture

IoT Architecture

A typical IoT system contains:

Device Layer

      ↓

Network Layer

      ↓

Processing Layer

      ↓

Application Layer

1. Device Layer

Contains physical hardware:

Sensors
Controllers
Embedded devices

Examples:

Temperature sensors
Cameras
Motion sensors

2. Network Layer

Transfers data between devices.

Technologies:

Wi-Fi
Bluetooth
Zigbee
5G

3. Processing Layer

Handles collected data.

Examples:

Edge computing
Cloud computing
Data processing

4. Application Layer

Provides user-facing applications.

Examples:

Mobile apps
Dashboards
Monitoring systems

IoT Components

An IoT device usually contains:

Sensor

   +

Microcontroller

   +

Communication Module

   +

Software

Sensors

Sensors collect information from the environment.

Examples:

Temperature Sensor

Measures temperature.

Motion Sensor

Detects movement.

Light Sensor

Measures light intensity.

Microcontrollers

Microcontrollers control IoT devices.

Examples:

Raspberry Pi
Arduino
ESP32

Python and Raspberry Pi

What is Raspberry Pi?

Raspberry Pi is a small computer used for:

Learning electronics
IoT projects
Automation

Installing Python on Raspberry Pi

Check Python:

python --version

GPIO Programming

GPIO allows Python to control hardware pins.

Library:

pip install RPi.GPIO

Controlling LED with Python

Example:

import RPi.GPIO as GPIO
import time


GPIO.setmode(GPIO.BOARD)


GPIO.setup(
11,
GPIO.OUT
)


GPIO.output(
11,
GPIO.HIGH
)


time.sleep(2)


GPIO.cleanup()

Reading Sensor Data

Example:

temperature = sensor.read()

print(
temperature
)

IoT Communication Protocols

Devices need communication methods.

1. MQTT

MQTT is a lightweight messaging protocol.

Used for:

Sensors
Smart devices
Real-time communication

MQTT Architecture:

Device

  ↓

MQTT Broker

  ↓

Application

2. HTTP

Used for web communication.

Example:

IoT Device

      ↓

REST API

      ↓

Server

3. Bluetooth

Used for:

Short-distance communication
Wearable devices

4. WebSockets

Used for:

Real-time updates
Live dashboards

Python MQTT Example

Install:

pip install paho-mqtt

Example:

import paho.mqtt.client as mqtt


client=mqtt.Client()


client.connect(
"broker"
)


client.publish(
"sensor",
"Temperature Data"
)

IoT Data Processing

IoT devices generate large amounts of data.

Python helps with:

Data cleaning
Analysis
Visualization

Libraries:

Pandas
NumPy
Matplotlib

IoT Data Flow

Sensor Data

     ↓

Python Processing

     ↓

Database

     ↓

Dashboard

IoT Databases

Common databases:

Traditional Databases

MySQL
PostgreSQL

Time-Series Databases

Used for sensor data.

Examples:

InfluxDB

IoT Cloud Platforms

Popular platforms:

AWS IoT

Provides:

Device management
Data processing
Cloud integration

Google Cloud IoT Solutions

Provides:

Data analytics
Cloud processing

Azure IoT

Provides:

Device monitoring
Security

Python Cloud IoT Example

Python can:

Send sensor data
Receive commands
Process device events

Edge Computing

What is Edge Computing?

Processing data near the device instead of sending everything to the cloud.

Benefits:

Faster response
Lower network usage
Better reliability

Example:

Camera

 ↓

Local AI Processing

 ↓

Only Important Data Sent

IoT Security

Security is critical because IoT devices are connected systems.

Important practices:

Device Authentication

Verify devices.

Data Encryption

Protect communication.

Secure Updates

Keep software updated.

Access Control

Limit permissions.

IoT with Artificial Intelligence

AI + IoT = Intelligent IoT

Examples:

Smart cameras
Predictive maintenance
Smart assistants

Machine Learning in IoT

Python can analyze sensor data to:

Predict failures
Detect patterns
Automate decisions

Real-World IoT Projects

Project 1: Smart Temperature Monitor

Features:

Read temperature
Store data
Display dashboard

Technologies:

Raspberry Pi
Python
Sensor

Project 2: Smart Home Automation

Features:

Control lights
Monitor devices
Remote access

Technologies:

Python
MQTT
IoT hardware

Project 3: Weather Monitoring System

Features:

Collect weather data
Store readings
Show reports

Skills:

Sensors
Data visualization

Project 4: Smart Agriculture System

Features:

Soil monitoring
Automatic irrigation
Weather tracking

Project 5: Industrial Monitoring System

Features:

Machine monitoring
Failure prediction
Alerts

IoT Career Opportunities

IoT Developer

Works on:

Device programming
Sensor integration

Embedded Python Developer

Works on:

Hardware software interaction

IoT Data Engineer

Works on:

Data pipelines
Analytics

IoT Security Engineer

Works on:

Device protection
Secure communication

IoT Learning Roadmap

Python

 ↓

Electronics Basics

 ↓

Raspberry Pi / Arduino

 ↓

Sensors

 ↓

Communication Protocols

 ↓

Cloud Integration

 ↓

AI + IoT

 ↓

IoT Projects

Practice Projects

Beginner:

LED controller
Temperature reader
Digital clock

Intermediate:

Smart home system
IoT dashboard
MQTT communication system

Advanced:

Industrial IoT platform
AI-powered IoT monitoring
Cloud-connected smart device

Chapter 75: Python Automation and Scripting Masterclass

Introduction

Automation means using programs to perform repetitive tasks automatically without manual effort.

Python is one of the best automation languages because it provides:

Simple syntax
Powerful libraries
Cross-platform support
Easy integration with other systems

Python automation is used in:

Software development
System administration
Data processing
Testing
Business operations
Cloud management

What is Python Automation?

Python automation means writing scripts that automatically perform tasks.

Examples:

Organizing files
Sending emails
Generating reports
Scraping websites
Managing servers
Automating testing

Automation Workflow

Identify Task

      ↓

Analyze Steps

      ↓

Write Python Script

      ↓

Test Script

      ↓

Schedule Automation

      ↓

Monitor Results

Types of Python Automation

Automation

     |

----------------------------

File     Web      System     Business

Auto     Auto     Auto       Auto

Section 1: File Automation

Python can automate:

Creating files
Moving files
Renaming files
Searching files
Organizing folders

Working with Files

Open file:

file=open(
"data.txt",
"r"
)

content=file.read()

print(content)

Creating a File

file=open(
"new.txt",
"w"
)

file.write(
"Hello Python"
)

file.close()

Using pathlib

Modern file handling:

from pathlib import Path


path=Path(
"example.txt"
)


print(
path.exists()
)

Organizing Files Automatically

Example:

import os
import shutil


source="downloads"


destination="images"


for file in os.listdir(source):

    if file.endswith(".jpg"):

        shutil.move(
        source+"/"+file,
        destination
        )

Section 2: Folder Automation

Python can:

Create folders
Delete folders
Search directories

Example:

import os


os.mkdir(
"Projects"
)

Section 3: System Automation

Python can automate operating system tasks.

Libraries:

os
subprocess
platform

Getting System Information

import platform


print(
platform.system()
)

print(
platform.processor()
)

Running System Commands

import subprocess


result=subprocess.run(
"dir",
shell=True
)


print(result)

Section 4: Task Scheduling

Automation often requires running tasks automatically.

Examples:

Daily reports
Backups
Data collection

Schedule Library

Install:

pip install schedule

Example:

import schedule
import time


def job():

    print(
    "Task Running"
    )


schedule.every().day.do(job)


while True:

    schedule.run_pending()

    time.sleep(1)

Section 5: Web Automation

Web automation controls browsers automatically.

Used for:

Testing websites
Data collection
Form filling

Selenium

Popular browser automation tool.

Install:

pip install selenium

Example:

from selenium import webdriver


browser=webdriver.Chrome()


browser.get(
"https://example.com"
)

Selenium Automation Tasks

Can automate:

Clicking buttons
Filling forms
Taking screenshots
Testing websites

Section 6: Web Scraping Automation

Web scraping collects information from websites.

Libraries:

Requests
BeautifulSoup

Install:

pip install beautifulsoup4

Example:

import requests
from bs4 import BeautifulSoup


response=requests.get(
"https://example.com"
)


soup=BeautifulSoup(
response.text,
"html.parser"
)


print(
soup.title
)

Section 7: API Automation

APIs allow programs to communicate.

Python can automate:

Data retrieval
Reports
Integrations

Example:

import requests


response=requests.get(
"https://api.example.com"
)


data=response.json()


print(data)

Section 8: Email Automation

Python can automate emails.

Uses:

SMTP
Email libraries

Example:

import smtplib


server=smtplib.SMTP(
"smtp.gmail.com",
587
)


server.starttls()

Applications:

Reports
Notifications
Alerts

Section 9: Excel Automation

Python can automate spreadsheets.

Library:

OpenPyXL
Pandas

Install:

pip install openpyxl

Example:

from openpyxl import Workbook


workbook=Workbook()


sheet=workbook.active


sheet["A1"]="Python"


workbook.save(
"file.xlsx"
)

Section 10: PDF Automation

Python can:

Create PDFs
Extract text
Generate reports

Libraries:

PyPDF
ReportLab

Section 11: Database Automation

Python can automate:

Data insertion
Backups
Reports

Example:

import sqlite3


connection=sqlite3.connect(
"data.db"
)


cursor=connection.cursor()

Section 12: Testing Automation

Automation testing checks software automatically.

Tools:

pytest
Selenium
unittest

Example:

def test_login():

    assert True

Section 13: DevOps Automation

Python helps automate:

Deployment
Server management
Cloud operations

Examples:

AWS automation
CI/CD scripts
Monitoring tools

Section 14: AI Automation

Python + AI can automate intelligent tasks.

Examples:

Chatbots
Document processing
Recommendation systems

Professional Automation Project Ideas

Project 1: Automatic Backup System

Features:

Backup files
Compress folders
Schedule backups

Skills:

File handling
Scheduling

Project 2: Automated Report Generator

Features:

Collect data
Create charts
Generate reports

Skills:

Pandas
Excel
PDF generation

Project 3: Website Monitoring Tool

Features:

Check website status
Send alerts

Skills:

Requests
Scheduling

Project 4: Social Media Automation Tool

Features:

Schedule posts
Generate reports

Skills:

APIs
Automation workflows

Project 5: Cloud Automation Tool

Features:

Manage cloud resources
Create backups
Monitor services

Skills:

Cloud APIs
Python scripting

Automation Best Practices

Write Reusable Scripts

Avoid repeating code.

Add Error Handling

Handle failures safely.

Create Logs

Track automation activity.

Protect Credentials

Use:

Environment variables
Secret managers

Document Scripts

Explain:

Purpose
Usage
Requirements

Automation Career Opportunities

Automation Engineer

Builds:

Workflow automation
Testing systems

DevOps Engineer

Builds:

Deployment automation
Infrastructure tools

Python Developer

Creates:

Scripts
Backend systems

RPA Developer

Creates:

Business process automation

Python Automation Learning Roadmap

Python Basics

      ↓

File Handling

      ↓

OS Automation

      ↓

Web Automation

      ↓

API Automation

      ↓

Task Scheduling

      ↓

Cloud Automation

      ↓

Professional Tools

Practice Projects

Beginner:

File organizer
Password generator
Automatic calculator

Intermediate:

Web scraper
Email automation
Report generator

Advanced:

Cloud automation platform
DevOps automation system
AI-powered automation tool

Chapter 76: Python Game Development with Pygame

Introduction

Python is not only used for web development, automation, and AI—it is also an excellent language for learning game development.

The most popular Python game development library is Pygame, which provides tools for:

Graphics
Animation
Sound
Keyboard and mouse input
Collision detection
Game physics
Window management

With Pygame, you can build games such as:

Snake
Tic-Tac-Toe
Pong
Tetris
Platform games
Space shooters
Puzzle games

What is Pygame?

Pygame is an open-source library built on top of the SDL (Simple DirectMedia Layer) library.

It helps developers create:

2D games
Educational games
Simulations
Interactive applications

Installing Pygame

Install using pip:

pip install pygame

Verify installation:

import pygame

print(pygame.ver)

Creating Your First Game Window

import pygame

pygame.init()

screen = pygame.display.set_mode((800, 600))

pygame.display.set_caption("My First Game")

Basic Game Window

+--------------------------------------+
|                                      |
|                                      |
|          Game Window                 |
|                                      |
|                                      |
+--------------------------------------+

Game Loop

Every Pygame application runs inside a game loop.

Start

   ↓

Handle Events

   ↓

Update Game

   ↓

Draw Screen

   ↓

Repeat

Basic Game Loop

running = True

while running:

    for event in pygame.event.get():

        if event.type == pygame.QUIT:

            running = False

pygame.quit()

Colors in Pygame

Colors use RGB values.

Examples:

Color	RGB
Black	(0, 0, 0)
White	(255, 255, 255)
Red	(255, 0, 0)
Green	(0, 255, 0)
Blue	(0, 0, 255)

Filling the Screen

screen.fill((0, 0, 0))

pygame.display.update()

Drawing Shapes

Rectangle

pygame.draw.rect(
    screen,
    (255, 0, 0),
    (100, 100, 150, 80)
)

Circle

pygame.draw.circle(
    screen,
    (0, 255, 0),
    (300, 200),
    50
)

Line

pygame.draw.line(
    screen,
    (255, 255, 255),
    (0, 0),
    (400, 300),
    3
)

Coordinates

Pygame uses a coordinate system.

(0,0)

+---------------------------->

|

|

|

v

X increases to the right.
Y increases downward.

Displaying Text

font = pygame.font.SysFont(None, 40)

text = font.render(
    "Hello Pygame",
    True,
    (255,255,255)
)

screen.blit(text, (100,100))

Images

Load an image:

player = pygame.image.load(
    "player.png"
)

screen.blit(player, (100,100))

Keyboard Input

keys = pygame.key.get_pressed()

if keys[pygame.K_LEFT]:

    print("Moving Left")

Mouse Input

mouse_x, mouse_y = pygame.mouse.get_pos()

print(mouse_x, mouse_y)

Moving an Object

x = 100

x += 5

Frame Rate

Games run many times every second.

Use a clock:

clock = pygame.time.Clock()

clock.tick(60)

60 means 60 FPS (Frames Per Second).

Animation

Animation is created by repeatedly updating positions.

Frame 1

↓

Frame 2

↓

Frame 3

↓

Smooth Movement

Collision Detection

Example:

player_rect = pygame.Rect(50,50,50,50)

enemy_rect = pygame.Rect(70,70,50,50)

print(
player_rect.colliderect(enemy_rect)
)

Playing Sounds

Load sound:

sound = pygame.mixer.Sound(
    "jump.wav"
)

sound.play()

Background Music

pygame.mixer.music.load(
    "music.mp3"
)

pygame.mixer.music.play(-1)

Sprites

Sprites represent game objects.

Examples:

Player
Enemy
Bullet
Coin

Sprite Group

group = pygame.sprite.Group()

Basic Player Class

class Player:

    def __init__(self):

        self.x = 100

        self.y = 100

Game States

Games usually have multiple screens.

Main Menu

     ↓

Playing

     ↓

Paused

     ↓

Game Over

Score System

Example:

score = 0

score += 10

Timer

time_left = 60

Saving High Scores

with open("score.txt", "w") as file:

    file.write("100")

Organizing Game Files

game/

├── main.py

├── player.py

├── enemy.py

├── assets/

│   ├── images/

│   ├── sounds/

│   └── fonts/

└── levels/

Optimizing Games

Tips:

Load images once.
Avoid unnecessary calculations.
Reuse objects.
Keep frame rate stable.

Popular Pygame Game Projects

Beginner

Guess the Number
Pong
Tic-Tac-Toe
Snake

Intermediate

Space Shooter
Brick Breaker
Racing Game
Platform Game

Advanced

RPG Game
Multiplayer Game
Tower Defense
Survival Game

Game Development Workflow

Idea

 ↓

Design

 ↓

Assets

 ↓

Programming

 ↓

Testing

 ↓

Release

 ↓

Updates

Career Opportunities

Python game development can lead to roles such as:

Game Developer
Gameplay Programmer
Simulation Developer
Educational Software Developer
Prototype Developer

Although large commercial games are often built with engines like Unity or Unreal Engine, Pygame is excellent for learning game programming and creating 2D games.

Practice Projects

Beginner

Snake Game
Tic-Tac-Toe
Pong Game
Memory Matching Game

Intermediate

Space Shooter
Endless Runner
Maze Game
Flappy Bird Clone

Advanced

Chess Game with AI
Tower Defense Game
RPG Adventure Game
Multiplayer Online Game

Chapter 77: Python GUI Development with Tkinter and CustomTkinter

Introduction

GUI (Graphical User Interface) applications allow users to interact with software using windows, buttons, menus, text boxes, and other visual elements instead of typing commands.

Python provides several GUI frameworks, with Tkinter being the standard library included with Python. For modern-looking interfaces, CustomTkinter builds on Tkinter and offers attractive, customizable widgets.

GUI applications include:

Calculator apps
Text editors
File managers
Login systems
Inventory software
Banking applications
Desktop dashboards

What is Tkinter?

Tkinter is Python’s built-in GUI library.

Features:

Easy to learn
Cross-platform
Included with Python
Good for desktop applications

Installing Tkinter

Tkinter is included with most Python installations.

Verify installation:

import tkinter

print("Tkinter Installed")

Your First Tkinter Window

import tkinter as tk

root = tk.Tk()

root.title("My First App")

root.geometry("500x300")

root.mainloop()

Window Structure

+----------------------------------+
| My First App                     |
|                                  |
|                                  |
|                                  |
|                                  |
+----------------------------------+

Important Window Methods

Method	Purpose
title()	Set window title
geometry()	Set window size
configure()	Change background
mainloop()	Start application

Labels

Labels display text.

import tkinter as tk

root = tk.Tk()

label = tk.Label(
    root,
    text="Welcome to Python GUI"
)

label.pack()

root.mainloop()

Buttons

Buttons execute functions.

import tkinter as tk

def hello():

    print("Hello")

root = tk.Tk()

button = tk.Button(
    root,
    text="Click Me",
    command=hello
)

button.pack()

root.mainloop()

Entry Widget

Used for user input.

entry = tk.Entry(root)

entry.pack()

Get text:

text = entry.get()

print(text)

Text Widget

For multiple lines of text.

text = tk.Text(
    root,
    height=10,
    width=40
)

text.pack()

Checkbutton

var = tk.BooleanVar()

check = tk.Checkbutton(
    root,
    text="Accept",
    variable=var
)

check.pack()

Radiobutton

Choose one option.

choice = tk.StringVar()

tk.Radiobutton(
    root,
    text="Male",
    variable=choice,
    value="Male"
).pack()

tk.Radiobutton(
    root,
    text="Female",
    variable=choice,
    value="Female"
).pack()

Listbox

Display multiple items.

listbox = tk.Listbox(root)

listbox.insert(1, "Python")

listbox.insert(2, "Java")

listbox.pack()

Combobox

Using ttk.

from tkinter import ttk

combo = ttk.Combobox(
    root,
    values=["Red", "Blue", "Green"]
)

combo.pack()

Message Box

from tkinter import messagebox

messagebox.showinfo(
    "Information",
    "Operation Successful"
)

File Dialog

Open a file chooser.

from tkinter import filedialog

filename = filedialog.askopenfilename()

print(filename)

Layout Managers

Tkinter provides three layout managers.

1. pack()

Places widgets one after another.

button.pack()

2. grid()

Places widgets in rows and columns.

label.grid(
    row=0,
    column=0
)

3. place()

Positions widgets using coordinates.

button.place(
    x=50,
    y=100
)

Event Handling

Respond to user actions.

Example:

def clicked():

    print("Button Pressed")

Keyboard Events

def key(event):

    print(event.keysym)

root.bind(
    "<Key>",
    key
)

Mouse Events

def mouse(event):

    print(event.x, event.y)

root.bind(
    "<Button-1>",
    mouse
)

Menus

menu = tk.Menu(root)

root.config(menu=menu)

Frames

Frames organize widgets.

frame = tk.Frame(root)

frame.pack()

Images

image = tk.PhotoImage(
    file="logo.png"
)

label = tk.Label(
    root,
    image=image
)

label.pack()

Canvas

Used for drawing graphics.

canvas = tk.Canvas(
    root,
    width=300,
    height=200
)

canvas.pack()

canvas.create_rectangle(
    50,
    50,
    150,
    120,
    fill="blue"
)

CustomTkinter

CustomTkinter provides a modern appearance for Tkinter applications.

Install:

pip install customtkinter

Creating a CustomTkinter Window

import customtkinter as ctk

app = ctk.CTk()

app.title("Modern App")

app.geometry("500x400")

app.mainloop()

Modern Button

button = ctk.CTkButton(
    app,
    text="Submit"
)

button.pack(pady=20)

Modern Entry

entry = ctk.CTkEntry(app)

entry.pack()

Appearance Mode

ctk.set_appearance_mode("Dark")

Options:

Light
Dark
System

Color Themes

ctk.set_default_color_theme(
    "blue"
)

Available themes:

blue
green
dark-blue

Login Form Example

+--------------------------+

Username

[____________]

Password

[____________]

[ Login ]

+--------------------------+

Building a Calculator

Components:

Buttons
Display
Events

Operations:

Addition
Subtraction
Multiplication
Division

Text Editor

Features:

Open files
Save files
Edit text
Search

Student Management System

Features:

Add students
Update records
Delete records
Search students

Database Integration

Tkinter works with databases.

Example:

import sqlite3

connection = sqlite3.connect(
    "students.db"
)

Best Practices

Use functions for event handling.
Separate GUI and business logic.
Validate user input.
Keep layouts organized.
Use meaningful widget names.

Project Folder Structure

gui_app/

├── main.py

├── database.py

├── models.py

├── views.py

├── controller.py

├── assets/

│   ├── icons/

│   └── images/

└── database.db

Popular Desktop Applications Built with Python

Examples include:

Text editors
Download managers
Accounting software
POS systems
Medical management software
Inventory systems
Personal productivity tools

Practice Projects

Beginner

Digital Calculator
To-Do List
Unit Converter
Temperature Converter

Intermediate

Notepad Application
Student Management System
Expense Tracker
Password Manager

Advanced

Inventory Management System
Banking Management Software
CRM Desktop Application
AI Desktop Assistant

Career Opportunities

GUI development skills are useful for:

Desktop Application Developer
Business Software Developer
Automation Tool Developer
Internal Enterprise Software Developer

Chapter 78: Python Networking and Socket Programming

Introduction

Networking allows computers and devices to communicate and exchange data over local networks or the internet.

Python provides the built-in socket module, making it easy to build:

Client-server applications
Chat applications
File transfer systems
Network monitoring tools
Web servers
Multiplayer games
IoT communication systems

Networking is one of the most valuable skills for backend development, cybersecurity, cloud computing, DevOps, and distributed systems.

What is Computer Networking?

A computer network is a group of connected devices that communicate using standard protocols.

Examples include:

Home Wi-Fi networks
Office LANs
The Internet
Cloud infrastructure

Basic Network Architecture

Client

   │

   ▼

Internet / LAN

   │

   ▼

Server

Types of Networks

LAN (Local Area Network)

Small geographical area.

Examples:

School
Office
Home

MAN (Metropolitan Area Network)

Covers a city or metropolitan area.

WAN (Wide Area Network)

Large geographical area.

Example:

Internet

Network Devices

Common networking devices include:

Router
Switch
Modem
Firewall
Access Point

IP Address

Every device on a network has an IP address.

Example:

192.168.1.10

Types:

IPv4
IPv6

Port Numbers

A port identifies a specific service running on a device.

Common ports:

Service	Port
HTTP	80
HTTPS	443
FTP	21
SSH	22
SMTP	25

Network Protocols

Protocols define communication rules.

Important protocols:

TCP
UDP
HTTP
HTTPS
FTP
SMTP
DNS

TCP Protocol

TCP (Transmission Control Protocol) provides:

Reliable communication
Ordered delivery
Error checking

Suitable for:

Websites
Email
Banking systems
File transfers

UDP Protocol

UDP (User Datagram Protocol) provides:

Fast communication
Lower overhead
No delivery guarantee

Suitable for:

Online gaming
Video streaming
Voice calls

TCP vs UDP

Feature	TCP	UDP
Reliable	✅	❌
Fast	❌	✅
Ordered Data	✅	❌
Error Recovery	✅	❌

Client-Server Model

Most applications follow this architecture.

Client

     ↓ Request

Server

     ↓ Response

Client

Examples:

Web browser → Website
Mobile app → API server
Chat app → Chat server

Python Socket Module

Import socket:

import socket

Create a socket:

server = socket.socket(
    socket.AF_INET,
    socket.SOCK_STREAM
)

Where:

AF_INET → IPv4
SOCK_STREAM → TCP

Creating a TCP Server

import socket

server = socket.socket(
    socket.AF_INET,
    socket.SOCK_STREAM
)

server.bind(("localhost", 5000))

server.listen()

print("Server started...")

Accepting Client Connections

client, address = server.accept()

print(address)

Receiving Data

message = client.recv(1024)

print(message.decode())

Sending Data

client.send(
    "Hello Client".encode()
)

Closing Connection

client.close()

server.close()

Creating a TCP Client

import socket

client = socket.socket(
    socket.AF_INET,
    socket.SOCK_STREAM
)

client.connect(("localhost", 5000))

Sending Message

client.send(
    "Hello Server".encode()
)

Receiving Reply

reply = client.recv(1024)

print(reply.decode())

Socket Communication Flow

Client

    │ Connect

    ▼

Server

    │ Accept

    ▼

Receive Message

    │

Send Response

    ▼

Client Receives Data

Building a Simple Echo Server

Server:

while True:

    data = client.recv(1024)

    client.send(data)

The server sends back exactly what it receives.

UDP Socket Example

Server:

server = socket.socket(
    socket.AF_INET,
    socket.SOCK_DGRAM
)

Receive data:

data, address = server.recvfrom(1024)

Hostname

Get local hostname:

import socket

print(socket.gethostname())

Local IP Address

import socket

hostname = socket.gethostname()

ip = socket.gethostbyname(hostname)

print(ip)

Multithreaded Server

A server should handle multiple clients simultaneously.

Architecture:

Server

 ├── Client 1 Thread

 ├── Client 2 Thread

 ├── Client 3 Thread

 └── Client 4 Thread

Thread Example

import threading

thread = threading.Thread(
    target=handle_client
)

thread.start()

File Transfer

Basic workflow:

Client

   │

Upload File

   ▼

Server

   │

Save File

Python can transfer files using sockets by reading and sending file bytes.

Chat Application

Architecture:

User A

     │

Chat Server

     │

User B

Features:

Multiple users
Real-time messaging
Username support

Network Timeouts

Prevent waiting forever.

client.settimeout(10)

Timeout after 10 seconds.

Exception Handling

try:

    client.connect(("localhost",5000))

except Exception as error:

    print(error)

Socket Methods

Method	Purpose
bind()	Assign address
listen()	Wait for clients
accept()	Accept connection
connect()	Connect to server
send()	Send data
recv()	Receive data
close()	Close socket

Security Considerations

Network applications should:

Validate incoming data
Encrypt sensitive communication
Authenticate users
Limit connection attempts
Handle errors safely

SSL/TLS

Secure communication uses encryption.

Client

     ↓

Encrypted Connection

     ↓

Server

HTTPS is HTTP running over TLS.

Socket Programming Best Practices

Close sockets properly.
Handle exceptions.
Use timeouts.
Validate input.
Log connection events.
Use encryption for sensitive data.

Real-World Networking Applications

Python networking is used in:

Chat applications
Multiplayer games
IoT systems
Monitoring software
Network scanners
Remote management tools
Proxy servers

Practice Projects

Beginner

Echo Server
Echo Client
Port Scanner (for systems you own or have permission to test)

Intermediate

Multi-user Chat Application
File Transfer System
Network Monitoring Tool
Remote Command Logger (for authorized environments)

Advanced

Multiplayer Game Server
Secure Chat Application
Distributed File Sharing System
Real-Time Notification Server

Career Opportunities

Networking knowledge is valuable for:

Backend Developer
Network Engineer
DevOps Engineer
Cloud Engineer
Cybersecurity Analyst
Systems Engineer
IoT Developer

Chapter 79: Python Multithreading, Multiprocessing, and Asynchronous Programming

Introduction

Modern applications often perform many tasks at the same time, such as:

Downloading files
Handling multiple users
Processing large datasets
Reading databases
Making API requests
Running background jobs

Python provides three major approaches for concurrent programming:

Multithreading – Multiple threads within a single process
Multiprocessing – Multiple processes running in parallel
Asynchronous Programming (asyncio) – Non-blocking execution using an event loop

Choosing the right approach depends on the type of work your program performs.

Understanding Concurrency

Concurrency means multiple tasks make progress during the same period.

Example:

Task A

Task B

Task C

↓

Tasks progress together

Understanding Parallelism

Parallelism means multiple tasks actually run at the same time on different CPU cores.

CPU Core 1 → Task A

CPU Core 2 → Task B

CPU Core 3 → Task C

Concurrency vs Parallelism

Feature	Concurrency	Parallelism
Tasks overlap	✅	✅
Runs simultaneously	Not always	Yes
Multiple CPU cores required	No	Usually

Process vs Thread

Process

A process is an independent running program.

Examples:

Chrome browser
VS Code
Python interpreter

Each process has its own memory.

Thread

A thread is a lightweight unit of execution inside a process.

A process can contain multiple threads.

Process

 ├── Thread 1

 ├── Thread 2

 └── Thread 3

When to Use Threads

Threads are ideal for I/O-bound tasks, such as:

Reading files
API requests
Database operations
Network communication

Python Threading Module

import threading

Creating a Thread

import threading


def hello():

    print("Hello")


thread = threading.Thread(
    target=hello
)

thread.start()

Waiting for a Thread

thread.join()

The program waits until the thread finishes.

Multiple Threads

for i in range(5):

    thread = threading.Thread(
        target=hello
    )

    thread.start()

Thread Lifecycle

Created

   ↓

Started

   ↓

Running

   ↓

Completed

Thread Synchronization

Multiple threads may access the same resource.

Example:

Thread A

      ↓

Shared Variable

      ↑

Thread B

This can cause inconsistent results.

Lock

A Lock ensures only one thread accesses a shared resource at a time.

import threading

lock = threading.Lock()

with lock:

    print("Protected")

Race Condition

A race condition occurs when multiple threads modify shared data simultaneously.

Example:

Counter = 5

Thread A updates

Thread B updates

↓

Unexpected result

Multiprocessing

Multiprocessing creates separate processes.

Each process has its own memory space.

Ideal for:

CPU-intensive calculations
Image processing
Scientific computing
Machine learning

Multiprocessing Module

from multiprocessing import Process

Creating a Process

from multiprocessing import Process


def work():

    print("Processing")


process = Process(
    target=work
)

process.start()

process.join()

Process Architecture

Main Process

      │

 ├── Process 1

 ├── Process 2

 └── Process 3

Threads vs Processes

Feature	Threads	Processes
Memory	Shared	Separate
Creation Speed	Fast	Slower
Communication	Easy	More complex
Best For	I/O tasks	CPU tasks

Inter-Process Communication (IPC)

Processes cannot directly share memory.

Common IPC methods:

Queue
Pipe
Shared memory

Queue Example

from multiprocessing import Queue

queue = Queue()

queue.put("Python")

print(queue.get())

Pool

A process pool manages multiple worker processes.

from multiprocessing import Pool


def square(x):

    return x*x


with Pool() as pool:

    result = pool.map(
        square,
        [1,2,3]
    )

Asynchronous Programming

Async programming executes tasks without blocking the program while waiting for operations such as network requests or file access.

Suitable for:

APIs
Web servers
Web scraping
Real-time applications

asyncio Module

import asyncio

Async Function

async def hello():

    print("Hello")

Running an Async Function

import asyncio


async def hello():

    print("Hello")


asyncio.run(
    hello()
)

await Keyword

await pauses the current coroutine until another asynchronous operation completes.

Example:

import asyncio


async def task():

    await asyncio.sleep(2)

    print("Done")

Event Loop

The event loop schedules and manages asynchronous tasks.

Event Loop

     │

 ├── Task A

 ├── Task B

 └── Task C

Running Multiple Tasks

import asyncio


async def task(number):

    print(number)


async def main():

    await asyncio.gather(

        task(1),

        task(2),

        task(3)

    )


asyncio.run(main())

Coroutine

A coroutine is an asynchronous function defined with async def.

Example:

async def download():

    pass

Blocking vs Non-Blocking

Blocking:

Task 1

↓

Wait

↓

Task 2

Non-Blocking:

Task 1

Task 2

Task 3

↓

Progress Together

Choosing the Right Approach

Task Type	Recommended Approach
Reading files	Threading
API requests	Asyncio
Web server	Asyncio
Image processing	Multiprocessing
Machine learning	Multiprocessing
Network communication	Threading or Asyncio

Common Mistakes

Avoid:

Creating unnecessary threads
Ignoring thread synchronization
Blocking inside async functions
Forgetting to close resources
Sharing mutable data without protection

Best Practices

Use threading for I/O-bound tasks.
Use multiprocessing for CPU-bound tasks.
Use asyncio for high-concurrency applications.
Protect shared resources with locks.
Keep concurrent code simple and well-documented.

Real-World Applications

Concurrency is used in:

Web servers
Chat applications
Cloud services
File synchronization
Data pipelines
Video processing
AI inference services

Practice Projects

Beginner

Multithreaded file downloader
Parallel number calculator
Async timer application

Intermediate

Async web scraper
Multi-client chat server
Image processing tool using multiprocessing

Advanced

High-performance REST API
Distributed task processing system
Real-time stock market data processor
Large-scale web crawler

Career Opportunities

Knowledge of concurrency is valuable for:

Backend Developer
Cloud Engineer
DevOps Engineer
AI/ML Engineer
Data Engineer
Systems Programmer
High-Performance Computing Engineer

Chapter 80: Python Design Patterns and Advanced Programming

Introduction

Professional software development requires more than writing code that works. Large applications need code that is:

Maintainable
Reusable
Flexible
Easy to test
Easy to extend

Design patterns are proven solutions to common software design problems.

Python supports many programming patterns because of its:

Object-oriented features
Dynamic nature
First-class functions
Powerful standard library

What Are Design Patterns?

A design pattern is a reusable approach for solving a common programming problem.

It is not a ready-made piece of code, but a design idea.

Example:

Problem

   ↓

Common Solution

   ↓

Reusable Pattern

Categories of Design Patterns

The classic patterns are divided into three groups:

Design Patterns

      |

----------------------------

Creational

Structural

Behavioral

1. Creational Patterns

Focus on object creation.

Examples:

Singleton
Factory
Builder
Prototype

2. Structural Patterns

Focus on object relationships.

Examples:

Adapter
Decorator
Facade
Proxy

3. Behavioral Patterns

Focus on communication between objects.

Examples:

Observer
Strategy
Command
Iterator

SOLID Principles

SOLID principles help create better object-oriented software.

S — Single Responsibility Principle

A class should have only one responsibility.

Bad:

class User:

    def save_database(self):
        pass

    def send_email(self):
        pass

The class does too many things.

Better:

User Class

Database Class

Email Class

O — Open/Closed Principle

Software should be:

Open for extension
Closed for modification

Example:

Add new features without changing existing code.

L — Liskov Substitution Principle

Child classes should be replaceable with parent classes.

Example:

class Animal:

    def sound(self):
        pass

Child classes should follow the same behavior.

I — Interface Segregation Principle

Do not force classes to implement unnecessary methods.

D — Dependency Inversion Principle

High-level modules should not depend directly on low-level modules.

Use abstractions.

Singleton Pattern

Ensures only one instance of a class exists.

Examples:

Database connection
Configuration manager

Example:

class Singleton:

    instance = None


    def __new__(cls):

        if cls.instance is None:

            cls.instance = super().__new__(cls)

        return cls.instance

Factory Pattern

Creates objects without exposing creation logic.

Example:

class Dog:

    def speak(self):

        return "Woof"


class Cat:

    def speak(self):

        return "Meow"


class AnimalFactory:

    def create(type):

        if type=="dog":

            return Dog()

        return Cat()

Builder Pattern

Creates complex objects step by step.

Useful for:

Reports
Configurations
Large objects

Prototype Pattern

Creates new objects by copying existing objects.

Useful when object creation is expensive.

Adapter Pattern

Allows incompatible objects to work together.

Example:

Old System

    ↓

Adapter

    ↓

New System

Decorator Pattern

Adds functionality without modifying original code.

Python decorators are based on this idea.

Example:

def logger(func):

    def wrapper():

        print("Running")

        func()

    return wrapper

Using Decorators

@logger
def hello():

    print("Hello")

Facade Pattern

Provides a simple interface to a complex system.

Example:

User

 ↓

Simple API

 ↓

Complex System

Proxy Pattern

Acts as a middle layer.

Used for:

Security
Caching
Access control

Observer Pattern

Objects automatically receive updates.

Example:

Notifications
Event systems

Architecture:

Subject

 ↓

Observers

Strategy Pattern

Allows selecting different algorithms dynamically.

Example:

Payment methods:

Credit Card

UPI

Bank Transfer

Command Pattern

Encapsulates actions as objects.

Used in:

Undo systems
Task queues

Iterator Pattern

Provides sequential access to elements.

Python uses iterators everywhere.

Example:

numbers=[1,2,3]

for n in numbers:

    print(n)

Advanced Object-Oriented Programming

Python supports advanced OOP features.

Abstract Classes

Abstract classes define required methods.

Example:

from abc import ABC, abstractmethod


class Shape(ABC):

    @abstractmethod
    def area(self):

        pass

Multiple Inheritance

Python allows multiple parent classes.

Example:

class A:
    pass


class B:
    pass


class C(A,B):
    pass

Method Resolution Order (MRO)

Python decides which method to call using MRO.

Check:

ClassName.mro()

Magic Methods

Special methods surrounded by double underscores.

Examples:

__init__()

__str__()

__len__()

__add__()

Operator Overloading

Example:

class Number:

    def __add__(self,other):

        return 10

Properties

Control attribute access.

Example:

class User:

    @property
    def name(self):

        return self._name

Class Methods

Operate on classes.

@classmethod
def create(cls):
    pass

Static Methods

Independent utility functions.

@staticmethod
def helper():
    pass

Metaclasses

A metaclass controls how classes are created.

Normal:

Object

 ↓

Class

 ↓

Metaclass

Example:

class Meta(type):

    pass

Uses of Metaclasses

Used in:

Framework development
ORMs
Validation systems

Examples:

Django models

Descriptors

Descriptors control attribute behavior.

Special methods:

__get__()

__set__()

__delete__()

Example uses:

Properties
ORM fields
Validation

Context Managers

Context managers manage resources automatically.

Example:

with open("file.txt") as file:

    data=file.read()

The file closes automatically.

Creating Custom Context Manager

class Manager:

    def __enter__(self):

        print("Start")


    def __exit__(self,*args):

        print("End")

Dependency Injection

Dependency Injection provides required objects from outside.

Without:

Class creates dependency

With:

Dependency provided externally

Benefits:

Easier testing
Flexible design

Clean Architecture

Professional applications separate responsibilities.

Example:

Presentation

      ↓

Business Logic

      ↓

Data Layer

      ↓

Database

MVC Pattern

Common in web applications.

Model

View

Controller

Python Framework Examples

Django

Uses:

MTV architecture

Flask

Uses:

Lightweight architecture

FastAPI

Uses:

API-based architecture

Advanced Python Programming Practices

Type Hints

Example:

def add(
a:int,
b:int
)->int:

    return a+b

Dataclasses

Simplify data objects.

from dataclasses import dataclass


@dataclass
class User:

    name:str

Enums

Represent fixed choices.

from enum import Enum

Professional Project Structure

project/

├── app/

│   ├── models/

│   ├── services/

│   ├── controllers/

│   └── utils/

├── tests/

├── config/

└── main.py

Practice Projects

Beginner

Build a class-based calculator
Create a plugin system
Implement decorators

Intermediate

Build a payment system using Strategy Pattern
Create a logging framework
Build a custom ORM model system

Advanced

Design a scalable backend architecture
Create your own Python framework
Build a dependency injection system

Career Applications

Design patterns are useful for:

Senior Python Developer
Software Architect
Backend Engineer
Framework Developer
System Designer

Chapter 81: Python Performance Optimization and Profiling

Introduction

Writing code that works is only the first step in professional software development. High-quality applications must also be:

Fast
Efficient
Scalable
Memory-friendly
Reliable under heavy workloads

Performance optimization means improving the speed and resource usage of a Python program.

Python performance optimization is important for:

Web applications
Data processing
Artificial intelligence
Automation systems
Cloud applications
Large-scale software

What is Performance Optimization?

Performance optimization improves:

Execution speed
CPU usage
Memory usage
Database operations
Network efficiency

Example:

Before optimization:

Process 1 million records

Time: 10 minutes

After optimization:

Process 1 million records

Time: 30 seconds

Performance Optimization Workflow

Identify Problem

      ↓

Measure Performance

      ↓

Find Bottleneck

      ↓

Optimize Code

      ↓

Test Again

      ↓

Deploy

What is a Bottleneck?

A bottleneck is the slowest part of a program.

Examples:

Slow database query
Inefficient loop
Large memory usage
Slow API request

Measuring Python Performance

Never optimize without measuring.

Tools:

time module
timeit
cProfile
line_profiler
memory_profiler

Using time Module

Example:

import time


start = time.time()


# Code here


end = time.time()


print(end-start)

timeit Module

Used for accurate benchmarking.

Example:

import timeit


result = timeit.timeit(
    "sum(range(100))",
    number=10000
)


print(result)

Profiling with cProfile

cProfile analyzes function execution.

Run:

python -m cProfile app.py

It shows:

Function calls
Execution time
Slow functions

Using profile Module

Example:

import cProfile


def calculate():

    for i in range(100000):

        pass


cProfile.run(
    "calculate()"
)

Finding Slow Code

Example:

Slow:

result=[]

for i in range(100000):

    result.append(i*i)

Better:

result=[
    i*i 
    for i in range(100000)
]

Algorithm Optimization

The algorithm has the biggest impact on performance.

Big O Notation

Measures algorithm efficiency.

Examples:

O(1)

Constant time.

data[0]

O(n)

Linear time.

for item in data:

    print(item)

O(n²)

Slow for large data.

Example:

for i in data:

    for j in data:

        print(i,j)

Choosing Better Algorithms

Example:

Searching a list:

Slow:

item in list

For large data:

Use:

set()

because lookup is faster.

List vs Set Performance

List:

numbers = [1,2,3,4]

Search:

3 in numbers

Time:

O(n)

Set:

numbers = {1,2,3,4}

Search:

3 in numbers

Average:

O(1)

Optimizing Loops

Avoid unnecessary work.

Bad:

for item in data:

    calculate_constant_value()

Better:

value=calculate_constant_value()

for item in data:

    use(value)

Using Built-in Functions

Python built-ins are optimized.

Slow:

total=0

for x in numbers:

    total+=x

Better:

total=sum(numbers)

Generator Optimization

Lists store everything in memory.

Example:

numbers=[
    x*x 
    for x in range(1000000)
]

Uses high memory.

Generator:

numbers=(
    x*x 
    for x in range(1000000)
)

Generates values when needed.

Memory Optimization

Memory problems can slow applications.

Checking Memory Usage

Tools:

memory_profiler
tracemalloc

tracemalloc Example

import tracemalloc


tracemalloc.start()


data=[i for i in range(100000)]


print(
tracemalloc.get_traced_memory()
)

Avoiding Memory Leaks

Common causes:

Unreleased objects
Large global variables
Unclosed files
Growing caches

Garbage Collection

Python automatically manages memory.

Module:

import gc

Example:

gc.collect()

Using Slots

Normal class:

class User:

    pass

Uses more memory.

With slots:

class User:

    __slots__=[
        "name",
        "age"
    ]

Reduces memory usage.

String Optimization

Slow:

result=""

for word in words:

    result+=word

Better:

result="".join(words)

Database Optimization

Database operations often become bottlenecks.

Use Indexing

Without index:

Search every record

With index:

Direct lookup

Avoid Too Many Queries

Bad:

Query

Query

Query

Query

Better:

Single optimized query

Use Connection Pooling

Instead of creating connections repeatedly:

Create once

Reuse connections

Caching

Caching stores frequently used data.

Examples:

Redis
Memory cache

Example:

Without cache:

Request

↓

Database

↓

Response

With cache:

Request

↓

Cache

↓

Response

Multithreading Optimization

Useful for I/O tasks:

APIs
Files
Network requests

Example:

threading.Thread()

Multiprocessing Optimization

Useful for CPU-heavy tasks:

Data processing
Machine learning

Example:

multiprocessing.Process()

Async Optimization

Useful for many network operations.

Example:

asyncio

Python Interpreter Optimization

Ways to improve speed:

Use latest Python version
Avoid unnecessary imports
Use optimized libraries
Reduce object creation

Using Faster Libraries

Examples:

Instead of:

Pure Python loops

Use:

NumPy
Pandas
C extensions

NumPy Optimization

Slow:

for i in range(100000):

    result.append(i*2)

Fast:

numpy_array*2

Code Optimization Techniques

Avoid Global Variables

Local variables are faster.

Reduce Function Calls

Function calls have overhead.

Use Appropriate Data Structures

Choose:

List
Set
Dictionary
Tuple

based on requirement.

Production Performance Monitoring

Applications should be monitored.

Tools:

Application logs
Metrics
Monitoring dashboards

Profiling Tools Summary

Tool	Purpose
time	Simple timing
timeit	Benchmarking
cProfile	Function profiling
line_profiler	Line analysis
memory_profiler	Memory tracking
tracemalloc	Memory allocation

Performance Optimization Checklist

✅ Measure first
✅ Find bottlenecks
✅ Improve algorithms
✅ Optimize database queries
✅ Reduce memory usage
✅ Use caching
✅ Use concurrency correctly
✅ Benchmark changes

Real-World Optimization Examples

Web Application

Optimization:

Database indexing
Caching
Async requests

Data Processing

Optimization:

NumPy
Vectorization
Multiprocessing

AI Applications

Optimization:

GPU usage
Efficient models
Batch processing

Practice Projects

Beginner

Benchmark different algorithms
Optimize file processing script
Compare list vs set performance

Intermediate

Optimize a database application
Build a caching system
Profile a web application

Advanced

High-performance API server
Large-scale data processing pipeline
Performance monitoring platform

Career Applications

Performance optimization skills are valuable for:

Senior Python Developer
Backend Engineer
Data Engineer
Machine Learning Engineer
Cloud Engineer
Software Architect

Complete Python Roadmap (Beginner to Advanced)

Stage 1: Python Basics

1. Print Function

2. Input Function

3. Variables

4. Data Types

5. Comments

Stage 2: Operators

Arithmetic Operators

Comparison Operators

Logical Operators

Assignment Operators

Stage 3: Type Conversion

Stage 4: Strings

String Formatting

Stage 5: Conditional Statements

if

if else

if elif else

Stage 6: Loops

for Loop

while Loop

break

continue

Stage 7: Functions (Very Important)

Simple Function

Function with Parameter

Function with Return

Default Parameter

Keyword Arguments

Variable-Length Arguments

Keyword Variable Arguments

Lambda Function

Recursive Function

Stage 8: Lists

Stage 9: Tuples

Stage 10: Sets

Stage 11: Dictionaries

Stage 12: File Handling

Stage 13: Exception Handling

Stage 14: Object-Oriented Programming (OOP)

Stage 15: Modules

Stage 16: Popular Built-in Functions

Stage 17: Advanced Python

Python if Statement

Example 1: Simple if

Example 2: Condition is False

Comparison Operators Used in if

Example 3: Using ==

Example 4: Using !=

Using User Input

Using Strings

Multiple Statements Inside if

Indentation in Python

Nested if

Practice Programs

Program 1: Check Positive Number

Program 2: Check Even Number

Program 3: Check Password

Program 4: Check Passing Marks

Key Points

Exercises

Python if...else Statement

Syntax

Flow Diagram

Example 1: Check Age

Output

Explanation

Example 2: Condition is False

Output

Example 3: Even or Odd

Output

Explanation

Example 4: Positive or Negative

Example 5: Password Check

Example

Example 6: Largest of Two Numbers

Example 7: Check Divisibility

Example 8: Boolean Values

Example 9: Compare Strings

Python `if` Statement

Example 1: Simple `if`

Comparison Operators Used in `if`

Example 3: Using `==`

Example 4: Using `!=`

Multiple Statements Inside `if`

Nested `if`

Python `if...else` Statement

Nested `if...else`

Chapter 6: `for` Loop

What is a `for` Loop?

Understanding `range()`

1. `range(stop)`

2. `range(start, stop)`

3. `range(start, stop, step)`

Nested `for` Loop

Using `enumerate()`

Using `zip()`

Chapter 7: Python `while` Loop

What is a `while` Loop?

How `while` Loop Works

`break` Statement

`continue` Statement

`pass` Statement

Nested `while` Loop