Skip to content
Work Practice Products Coding interviews Machine learning coding Quant developer HFT course Blog Method Why me About Enquire Book a call

Coding · Graphs

Topological sort

Lesson · 12 min read · Graphs · Topological sort

Put the nodes of a directed graph in a line so every edge points forwards. It exists exactly when there is no cycle, and two linear-time algorithms find it.

The problem

A pricing library is built from modules, and some modules need others to be built first: the yield curve needs the date conventions, the pricer needs both the curve and the volatility surface, the risk report needs the pricer and the Greeks. In what order can the modules be built?

datesratescurvevolpricergreeksreport
An arrow u → v means u must be built before v.

A topological order is a sequence of all the nodes in which every edge points forwards: whenever there is an edge u → v, u comes before v. Build orders, course plans, spreadsheet recalculation and the order of steps in a data pipeline are all topological orders.

When an order exists

If the graph has a cycle, no order exists: each node on the cycle must come before the next, and going round the cycle that means a node before itself. Without a cycle an order always exists, because a directed graph with no cycle always has a node with no incoming edge (follow edges backwards from anywhere; without a cycle you must stop somewhere). Put that node first, remove it, and repeat.

abc
a before b, b before c, c before a: no line can satisfy all three.

So topological sorting doubles as a cycle test, and there is usually more than one valid order: in the library, "dates" and "rates" can be built in either order.

Kahn's algorithm

That argument is an algorithm. Count each node's incoming edges (its in-degree). Every node with in-degree 0 can go next. Placing one removes its outgoing edges, which may bring other nodes to zero.

Kahn's algorithm

topoSort(graph)

  1. Compute the in-degree of every node.
  2. Put every node with in-degree 0 in a queue.
  3. While the queue is not empty: take u from it and append it to the order.
  4. For each edge u → v: subtract 1 from the in-degree of v; if it reaches 0, add v to the queue.
  5. If the order holds every node, return it. Otherwise the graph has a cycle.

Using a min-heap instead of a queue gives the smallest order in dictionary order, which makes the answer unique; the step-by-step below does that. On the library, numbered 0 to 6 in the order of the figure's names:

Kahn's algorithm with the smallest ready node taken first. The tag on each node is its remaining in-degree.

  1. Step 1 / 8
    0in 01in 02in 13in 24in 25in 16in 2
    ready (in-degree 0)
    0011
    order
    —0

    Count incoming edges. 0, 1 have none, so they are ready.

  2. Step 2 / 8
    0in 01in 02in 03in 14in 25in 16in 2
    ready (in-degree 0)
    1021
    order
    00

    Take 0, the smallest ready node, and remove its outgoing edges. 2 now has no incoming edge and is ready.

  3. Step 3 / 8
    01in 02in 03in 04in 25in 16in 2
    ready (in-degree 0)
    2031
    order
    0011

    Take 1, the smallest ready node, and remove its outgoing edges. 3 now has no incoming edge and is ready.

  4. Step 4 / 8
    012in 03in 04in 15in 16in 2
    ready (in-degree 0)
    30
    order
    001122

    Take 2, the smallest ready node, and remove its outgoing edges.

  5. Step 5 / 8
    0123in 04in 05in 06in 2
    ready (in-degree 0)
    4051
    order
    00112233

    Take 3, the smallest ready node, and remove its outgoing edges. 4, 5 now have no incoming edge and are ready.

  6. Step 6 / 8
    01234in 05in 06in 1
    ready (in-degree 0)
    50
    order
    0011223344

    Take 4, the smallest ready node, and remove its outgoing edges.

  7. Step 7 / 8
    012345in 06in 0
    ready (in-degree 0)
    60
    order
    001122334455

    Take 5, the smallest ready node, and remove its outgoing edges. 6 now has no incoming edge and is ready.

  8. Step 8 / 8
    0123456in 0
    ready (in-degree 0)
    empty0
    order
    00112233445566

    Take 6, the smallest ready node, and remove its outgoing edges. Every node is placed: 0, 1, 2, 3, 4, 5, 6 respects every edge.

just placedreadyplaced

The order 0, 1, 2, 3, 4, 5, 6 is dates, rates, curve, vol, pricer, greeks, report.

The depth-first method

The other standard method comes from the previous module. In a depth-first search, a node finishes (turns black) only after everything reachable from it has finished. So if there is an edge u → v, v finishes before u. Listing nodes in order of finishing therefore gives every edge pointing backwards, and reversing that list gives a topological order.

def topo_dfs(n, adj):
    colour = [0] * n                 # 0 white, 1 grey, 2 black
    finished = []
    def visit(u):
        colour[u] = 1
        for v in adj[u]:
            if colour[v] == 1:
                raise ValueError("cycle")
            if colour[v] == 0:
                visit(v)
        colour[u] = 2
        finished.append(u)           # u finishes after everything it points to
    for s in range(n):
        if colour[s] == 0:
            visit(s)
    return finished[::-1]

Both run in O(n + m). Kahn's algorithm is iterative, detects a cycle by counting, and adapts easily (the smallest order, or the number of rounds when everything ready runs in parallel). The depth-first version is shorter when you already have a depth-first search written.

TimeO(n + m)
SpaceO(n + m)

With a heap for the smallest order, O(m + n log n).

Implementation

The problem at the end of this module asks for the smallest valid order of courses, so it uses a min-heap. In all six languages:

import heapq

def course_order(n, prereqs):
    after = [[] for _ in range(n)]       # b -> courses that need b
    indeg = [0] * n
    for a, b in prereqs:
        after[b].append(a)
        indeg[a] += 1
    ready = [c for c in range(n) if indeg[c] == 0]
    heapq.heapify(ready)                 # smallest available course first
    order = []
    while ready:
        c = heapq.heappop(ready)
        order.append(c)
        for nxt in after[c]:
            indeg[nxt] -= 1
            if indeg[nxt] == 0:
                heapq.heappush(ready, nxt)
    return order if len(order) == n else []   # short: a cycle blocked the rest

The same code is the reference solution to An order to take every course, where it is compiled and run against the tests in all six languages.

Watch the direction of the edges. In the course problems a pair [a, b] means b comes before a, so the edge is b → a. Reversing it gives a valid order of the wrong graph, which passes a symmetric example and fails the rest.