This is an unofficial free book created for educational purposes and is not affiliated with official algorithm groups or companies or Stack Overflow. The information presented in this book is not guaranteed to be correct or accurate, use it at your own risk.
Getting started with algorithms
A sample algorithmic problem
Getting Started with Simple Fizz Buzz Algorithm in Swift
It would then run through the numbers again to check if the numbers are divisible by 3 and 5. Essentially, the code would have to run through each number in the array twice - first run the numbers by 3 and then run it by 5.
Algorithm Complexity
Big-Theta notation
Comparison of the asymptotic notations
Big-Omega Notation
Another example to solve perfect matching algorithm: If number of vertices is odd then output "No Perfect Matching" otherwise try all possible matches. We'd like to say that the algorithm takes exponential time, but actually you can't prove an Ω(n^2) lower bound using the usual definition of Ω, since the algorithm runs in linear time for n odd.
Big-O Notation
Quick note: A fast algorithm is one that performs few operations, so if the number of operations grows to infinity faster, then the algorithm is slower: O(n) is better than O(n^2). For this, we consider the number of bytes in memory occupied by the algorithm as a function of the size of the input, and use Big-O in the same way.
A Simple Loop
When we count operations, we usually consider the worst case: for example, if we have a loop that can be executed at most n times and contains 5 operations, the number of operations we count is 5n.
A Nested Loop
We've clearly lowered the complexity, since we've roughly divided the number of operations we do by 2, but we're still in the same complexity class as defined by Big-O. To lower the complexity to a lower class, we will have to divide the number of operations by something that tends to infinity with n.
O(log n) types of Algorithms
So the operations inside the inner loop are executed n^2 times, the operations in the outer loop are executed n times, and the assignment to i is executed once. We can start from i + 1 in the inner loop, because all elements before it will already have been checked against all elements of the array, including the one at index i + 1.
An O(log n) example
Trees
Typical anary tree representation
Introduction
To check if two Binary trees are same or not
Binary Search Trees
Binary Search Tree - Insertion (Python)
Binary Search Tree - Deletion(C++)
Lowest common ancestor in a BST
Binary Search Tree - Python
Check if a tree is BST or not
Algorithm to check if a given binary tree is BST
If a given input tree follows Binary search tree property or not
Binary Tree traversals
Level Order traversal - Implementation
Pre-order, Inorder and Post Order traversal of a Binary Tree
Traversal (root) after the order is traversal of the left subtree of the node, then the right subtree, and then the node.
Lowest common ancestor of a Binary Tree
Finding lowest common ancestor
Graph
Storing Graphs (Adjacency Matrix)
One thing to note is that, if the graph is undirected, the matrix becomes symmetric. The only benefit is that we can easily find the relationship between u-v nodes and their cost using Adjacency Matrix.
Introduction To Graph Theory
In directed graphs, the edges have direction signs on one side, which means the edges are unidirectional. On the other hand, the edges of undirected graphs have direction signs on both sides, which means they are Bidirectional.
Storing Graphs (Adjacency List)
Procedure Adjacency-List(maxN, E): // maxN indicates the maximum number of nodes edge[maxN] = Vector() // E indicates the number of edges. From this we can easily find out the total number of nodes connected to any node and what those nodes are.
Topological Sort
But if we needed to find out if there is an edge between u and v, it would have been easier if we had an adjacency matrix.
Detecting a cycle in a directed graph using Depth First Traversal
Time complexity of search is O(V+E) where V is the number of vertices and E is the number of edges.
Thorup's algorithm
The root is the entire graph and the leaves are single vertices in the original graph (with the level of negative infinity). The distance of a component with more than one vertices is the minimum distance of its unexpanded children.
Graph Traversals
Depth First Search traversal function
Dijkstra’s Algorithm
Dijkstra's Shortest Path Algorithm
Now, we go to the other nodes from the source and if we can update them, then we will push them to the queue. We can clearly see that 5 is not the shortest distance we can travel to get to node 3.
A* Pathfinding
Introduction to A*
A* Pathfinding through a maze with no obstacles
Let's also assume that we cannot move diagonally to go from green to red. To calculate this heuristic, we will use this formula: distance = abs(from.x - to.x) + abs(from.y - to.y).
Solving 8-puzzle problem using A* algorithm
First, we find the heuristic value required to reach the final state from the initial state. The above value is obtained since 1 in the current state is 1 horizontal distance away from 1 in the end state.
A* Pathfinding Algorithm
Simple Example of A* Pathfinding: A maze with no obstacles
Since Down has a higher priority than Left, we choose the square that takes us "Down". Finally we can see that we have a winning field next to us, so we go there and we're done.
Dynamic Programming
Edit Distance
Weighted Job Scheduling Algorithm
We will have an additional temporary Acc_Prof array of size n (here n denotes the total number of tasks). The total amount of profit we can make if we choose these two jobs is: Acc_Prof[j].
Longest Common Subsequence
Now, if we want to find out which jobs were executed to get the maximum profit, we will traverse the array in reverse order, and if Acc_Prof matches maxProfit, we will push the name of the job onto a stack and subtract the Profit of that job from maxProfit. We will do this until our maxProfit > 0 or we reach the starting point of the Acc_Prof array.
Fibonacci Number
Longest Common Substring
Applications of Dynamic Programming
Fibonacci Numbers
As mentioned above, the iterative dynamic programming approach starts from the base cases and works up to the end result. The key observation to make to get the space complexity down to O(1) (constant) is the same observation we made for the recursive stack - we only need fibonacci(n-1) and fibonacci(n-2) to build fibonacci(n).
Kruskal's Algorithm
Optimal, disjoint-set based implementation
Simple, more detailed implementation
Simple, disjoint-set based implementation
Simple, high level implementation
Greedy Algorithms
Human Coding
The process of decompression is simply a matter of translating the stream of prefix codes into individual byte values, usually by traversing the Hufman tree node by node as each bit is read from the input stream. Usually, the Huffman tree is constructed using statistically adjusted data for each compression cycle, so the reconstruction is quite simple.
Activity Selection Problem
Sort the activity by end time, which means the activity ends first, first come first served. If the activity to be performed does not share a common time with the activities previously 2.
Change-making problem
Set: current_activity:= first activity 3. set: end time:= end time of current activity 4. go to the next activity if present, end if not existing. if start time of current activity <= end time: run the activity and go to 4 6. see here for coding help http://www.geeksforgeeks.org/greedy-algorithms-set-1-activity-selection-problem/ . let change_make money_system amount = let rec loop n = . let one piece acc piece = match n - piece with. if we don't reach 0, we discard this solution*) None.
Applications of Greedy technique
Oine Caching
For example, the LFU strategy needs to update the hit frequency for the cache pages, while the LFD strategy needs to recalculate the cache page distances. In the case of LRU, the strategy is independent of what is on the cache side, it only cares about the last use.
Ticket automat
The proof is quite long and can be found here or in the book by Jon Kleinberg and Eva Tardos (see sources in notes below). The optimal solution is clearly two coins of value 3, but greedy chooses 4 in the first step, so it must choose 1 in steps two and three.
Interval Scheduling
Suppose that greed is not optimal and i1,i2,..,ik denote the set of jobs chosen by the greedy. This second example shows that there are usually many possible greedy strategies, but only a few or even none can find the optimal solution in every case.
Minimizing Lateness
Of course, if a schedule has an inversion, one with a pair of consecutively scheduled inverted tasks has. Suggestion: Swapping two adjacent, inverted tasks reduces the number of inversions by one and does not increase the maximum delay.
Prim's Algorithm
Introduction To Prim's Algorithm
This time we consider node-1, node-2 and node-5 and take the smallest edge which is 5-4. We will select edges by adding a new node to our subgraph.
Bellman–Ford Algorithm
Single Source Shortest Path Algorithm (Given there is a negative cycle in a graph)
We want to discover the shortest path from the source to all other vertices. Since we need to relax the edges at most (V-1) times, the time complexity of this algorithm will be O(V * E), where E denotes the number of edges, if we use the adjacency list to represent the graph.
Detecting Negative Cycle in a Graph
Using the Bellman-Ford algorithm, we can detect if there is a negative cycle in our graph. This is because for each iteration, going through the negative loop always lowers the cost of the shortest path.
Why do we need to relax all the edges at most (V-1) times
Our very first iteration has found the shortest path from the source to all the other nodes. We can conclude that at best it takes 1 iteration to find the shortest path from the source.
Line Algorithm
Bresenham Line Drawing Algorithm
Floyd-Warshall Algorithm
All Pair Shortest Path Algorithm
Once we find you, we print you and start popping elements from the stack and printing them. To find out if there is a negative edge cycle, we will need to check the main diagonal of the distance matrix.
Catalan Number Algorithm
Catalan Number Algorithm Basic Information
Multithreaded Algorithms
Square matrix multiplication multithread
Multiplication matrix vector multithread
Knuth Morris Pratt (KMP) Algorithm
KMP-Example
Edit Distance Dynamic Algorithm
Minimum Edits required to convert string 1 to string 2
Online algorithms
If there is no constant r for which an online algorithm A is r-competitive, we call A non-competitive.
Paging (Online Caching)
Proof: Let σ be an instance of the paging problem and l the number of phases for σ. Now we try to show that the optimal offline algorithm exhibits at least k+l-2 pages for σ, k in the first phase and at least one for each subsequent phase except the last one.
Sorting
Bubble Sort
Bubble Sort
Implementation in C & C++
Implementation in C#
Python Implementation
Implementation in Java
Implementation in Javascript
Merge Sort
Merge Sort Basics
Merge Sort Implementation in Go
Merge Sort Implementation in C & C#
Merge Sort Implementation in Java
Merge Sort Implementation in Python
Bottoms-up Java Implementation
Insertion Sort
Haskell Implementation
Bucket Sort
C# Implementation
Quicksort
Quicksort Basics
Quicksort in Python
Lomuto partition java implementation
Counting Sort
Counting Sort Basic Information
Psuedocode Implementation
Heap Sort
C# Implementation
Heap Sort Basic Information
Cycle Sort
Pseudocode Implementation
Odd-Even Sort
Odd-Even Sort Basic Information
Selection Sort
Elixir Implementation
Selection Sort Basic Information
Implementation of Selection sort in C#
Searching
Binary Search
Rabin Karp
While calculating hash value, we divide it by a prime number to avoid collision. we get a match we have to check it character by character to make sure we got a proper match. This is to recalculate the hash value for pattern, first by removing the leftmost character and then adding the new character from the text.
Analysis of Linear search (Worst, Average and Best Cases)
In worst-case analysis, we calculate upper bounds on the running time of an algorithm. In the best case, we calculate a lower bound for the running time of an algorithm.
Binary Search: On Sorted Numbers
Linear search
Substring Search
Introduction To Knuth-Morris-Pratt (KMP) Algorithm
To achieve this, we will look for a suffix in our Pattern just before our mismatch occurred (substring abc), which is also a prefix of the substring of our Pattern. Here, a non-zero value of S[i] means that there is an S[i] length suffix the same as the prefix in that substring (substring from 0 to i) and the next comparison will start from S[i] + 1 position of the Pattern.
Introduction to Rabin-Karp Algorithm
If the rolling hash matches the hash value of our pattern, we will check whether the strings match or not. We subtract the value of the First Letter of Previous String from our current hash value.
Python Implementation of KMP algorithm
Because of the abundance of the searched strings, single-string search algorithms are impractical here. If we want to find any of the large number, say k, fixed length patterns in a text, we can create a simple variant of the Rabin-Karp.
KMP Algorithm in C
Breadth-First Search
Finding the Shortest Path from Source to other Nodes
If you haven't noticed, the node level simply indicates the shortest path distance from the source. This means that for each node we will work with, we will push all other directly traversable and not yet traversable nodes in the queue.
Finding Shortest Path from Source in a 2D graph
Connected Components Of Undirected Graph Using BFS
Depth First Search
Introduction To Depth-First Search
Hash Functions
Hash codes for common types in C#
Introduction to hash functions
Travelling Salesman
Brute Force Algorithm
Dynamic Programming Algorithm
Knapsack Problem
Knapsack Problem Basics
Solution Implemented in C#
Equation Solving
Linear Equation
Non-Linear Equation
Longest Common Subsequence
Longest Common Subsequence Explanation
Longest Increasing Subsequence
Longest Increasing Subsequence Basic Information
Check two strings are anagrams
Sample input and output
Generic Code for Anagrams
Pascal's Triangle
Pascal triangle in C
Algo:- Print a m*n matrix in square wise
Sample Example
Write the generic code
Matrix Exponentiation
Matrix Exponentiation to Solve Example Problems
Algorithm Pseudo Code
Dynamic Time Warping
Introduction To Dynamic Time Warping
Fast Fourier Transform
Radix 2 FFT
Radix 2 Inverse FFT
