text:
Table of Contents

Introduction

Chapter 1: General Introduction to Graph Partitioning

1.1. Partitioning

1.2. Mathematical notions

1.3. Graphs

1.4. Formal description of the graph partitioning problem

1.5. Objective functions for graph partitioning

1.6. Constrained graph partitioning

1.7. Unconstrained graph partitioning

1.8. Differences between constrained and unconstrained partitioning

1.9. From bisection to _k_ -partitioning: the recursive bisection method

1.10. NP-hardness of graph partitioning optimization problems

1.11. Conclusion

1.12. Bibliography

Part 1: Graph Partitioning for Numerical Analysis

Chapter 2: A Partitioning Requiring Rapidity and Quality: The Multilevel Method and Partitions Refinement Algorithms

2.1. Introduction

2.2. Principles of the multilevel method

2.3. Graph coarsening

2.4. Partitioning of the coarsened graph

2.5. Uncoarsening and partitions refinement

2.6. The spectral method

2.7. Conclusion

2.8. Bibliography

Chapter 3: Hypergraph Partitioning

3.1. Definitions and metrics

3.2. Connections between graphs, hypergraphs, and matrices

3.3. Algorithms for hypergraph partitioning

3.4. Purpose

3.5. Conclusion

3.6. Software references

3.7. Bibliography

Chapter 4: Parallelization of Graph Partitioning

4.1. Introduction

4.2. Distributed data structures

4.3. Parallelization of the coarsening phase

4.4. Folding

4.5. Centralization

4.6. Parallelization of the refinement phase

4.7. Experimental results

4.8. Conclusion

4.9. Bibliography

Chapter 5: Static Mapping of Process Graphs

5.1. Introduction

5.2. Static mapping models

5.3. Exact algorithms

5.4. Approximation algorithms

5.5. Conclusion

5.6. Bibliography

Part 2: Optimization Methods for Graph Partitioning

Chapter 6: Local Metaheuristics and Graph Partitioning

6.1. General introduction to metaheuristics

6.2. Simulated annealing

6.3. Iterated local search

6.4. Other local search metaheuristics

6.5. Conclusion

6.6. Bibliography

Chapter 7: Population-based Metaheuristics, Fusion-Fission and Graph Partitioning Optimization

7.1. Ant colony algorithms

7.2. Evolutionary algorithms

7.3. The fusion-fission method

7.4. Conclusion

7.5. Acknowledgments

7.6. Bibliography

Chapter 8: Partitioning Mobile Networks into Tariff Zones

8.1. Introduction

8.2. Spatial division of the network

8.3. Experimental results

8.4. Conclusion

8.5. Bibliography

Chapter 9: Air Traffic Control Graph Partitioning Application

9.1. Introduction

9.2. The problem of dividing up the airspace

9.3. Modeling the problem

9.4. Airspace partitioning: towards a new optimization metaheuristic

9.5. Division of the central European airspace

9.6. Conclusion

9.7. Acknowledgments

9.8. Bibliography

Part 3: Other Approaches to Graph Partitioning

Chapter 10: Application of Graph Partitioning to Image Segmentation

10.1. Introduction

10.2. The image viewed in graph form

10.3. Principle of image segmentation using graphs

10.4. Image segmentation via maximum flows

10.5. Unification of segmentation methods via graph theory

10.6. Conclusions and perspectives

10.7. Bibliography

Chapter 11: Distances in Graph Partitioning

11.1. Introduction

11.2. The Dice distance

11.3. Pons-Latapy distance

11.4. A partitioning method for distance arrays

11.5. A simulation protocol

11.6. Conclusions

11.7. Acknowledgments

11.8. Bibliography

Chapter 12: Detection of Disjoint or Overlapping Communities in Networks

12.1. Introduction

12.2. Modularity of partitions and coverings

12.3. Partitioning method

12.4. Overlapping partitioning methods

12.5. Conclusion

12.6. Acknowledgments

12.7. Bibliography

Chapter 13: Multilevel Local Optimization of Modularity

13.1. Introduction

13.2. Basics of modularity

13.3. Modularity optimization

13.4. Validation on empirical and artificial graphs

13.5. Discussion

13.6. Conclusion

13.7. Acknowledgments

13.8. Bibliography

Appendix. The Main Tools and Test Benches for Graph Partitioning

A.1. Tools for constrained graph partitioning optimization

A.2. Tools for unconstrained graph partitioning optimization

A.3. Graph partitioning test benches

A.4. Bibliography

Glossary

List of Authors

Index

First published 2011 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd
27-37 St George's Road
London SW19 4EU
UK | John Wiley & Sons, Inc.
111 River Street
Hoboken, NJ 07030
USA
---|---
www.iste.co.uk | www.wiley.com

© ISTE Ltd 2011

The rights of Charles-Edmond Bichot and Patrick Siarry to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

* * *

Library of Congress Cataloging-in-Publication Data

Graph partitioning / edited by Charles-Edmond Bichot, Patrick Siarry.

p. cm.

Includes bibliographical references and index.

ISBN 978-1-84821-233-6

1. Partitions (Mathematics) 2. Graph theory. I. Bichot, Charles-Edmond. II. Siarry, Patrick.

QA76.165.G73 2011

512.7'3--dc23

2011028388

* * *

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library

ISBN 978-1-84821-233-6

* * *
Introduction

Since the early 1970s, graph partitioning has been a widely researched topic. After 40 years of development, it is now time to evaluate the situation of the work done in this area.

Graph partitioning is a problem transverse to several fields in engineering, as well as research. The diversity of backgrounds of people working on this subject may explain why no real international community exists, despite the fact that some workshops have been organized on this subject over different periods of time. However, the number of people who work or have worked on this topic is quite significant, as shown by the abundant literature cited throughout the chapters of this book. Confronted with this profusion and diversity, it is interesting to gather the knowledge accumulated over so many years in order to synthesize it and draw the common theoretical fundamentals of this field.

This book intends to present to the neophyte reader, as well as the expert in applied mathematics or the computer science researcher, tools and methods to solve graph partitioning optimization problems. For this purpose, we have collected several methodological chapters detailing different graph partitioning optimization approaches such as the multilevel method, metaheuristics, partitioning parallelization, and hypergraph partitioning. In order to complete this theoretical part, several graph partitioning applications have been described on subjects as diverse as mobile networks, image segmentation, air traffic control, social networks, etc.

Despite the large number of studies in the domain of graph partitioning, it is clear that a lot of work remains to be done to solve this problem more efficiently. Recent years have seen the sizes of graphs to be partitioned soaring from a few thousand vertices to several million, or even billions of vertices. We hope that by reading this book the reader will feel inspired not only to take an interest in this problem, but also try to solve it more efficiently, both in terms of quality of partitions found and computation time required.

This book has three parts: Part 1 is dedicated to the most common application of graph partitioning, numerical analysis; Part 2 describes and implements several combinatorial optimization methods for graph partitioning; Part 3 presents other uses of graph partitioning.

Chapter 1 provides a general introduction to this book and is therefore independent of any part. It analyzes graph partitioning in order to identify the different problems associated with it.

A large number of studies on graph partitioning have been undertaken within the domain of numerical analysis. Part 1 of this book is dedicated to their presentation. Thus, Chapter 2 describes the methods and algorithms commonly used in numerical analysis to solve graph partitioning problems: the multilevel method, refinement algorithms like the Kernighan-Lin or Fiduccia-Mattheyses algorithms, and the spectral method. Chapter 3 introduces the particular case of hypergraph partitioning, which often occurs in numerical analysis. Chapter 4 presents several parallel algorithms to partition a graph. Finally, Chapter 5 presents the problem of static mapping which occurs in parallel computing.

Graph partitioning is often studied through a combinatorial optimization point of view. This is the theme of Part 2 of this book, which is dedicated to the study of combinatorial optimization methods, and more particularly metaheuristics, for graph partitioning. This part consists of two theoretical chapters followed by two application chapters. Chapter 6 focuses on the use of several local metaheuristics, like simulated annealing or iterated local search. Chapter 7 provides details on the use of population-based metaheuristics to optimize partitioning. This chapter explains the work done on Ant Colony algorithms and describes several adaptations of Genetic Algorithms to graph partitioning. It also introduces a recent method for graph partitioning optimization, fusion-fission, which works as a meta-method that overlooks a multilevel algorithm. The last 2 chapters of this part provide application examples for this part. Chapter 8 applies a Genetic Algorithm to the problem of mobile network partitioning in tariff zones. Chapter 9 describes an Air Traffic Control problem and solves it using the fusion-fission method.

Part 3 of this book develops other approaches of graph partitioning. In this part, we have chosen to focus on graph partitioning optimization, and thus we limited the scope of this part, even if many other works on graph partitioning could have been added, like constraint programming for graph partitioning or graph decomposition. Chapter 10 outlines the image segmentation problem and offers several methods to solve this problem, based on graph partitioning. Chapter 11 compares several distances between vertices, in order to build communities, using classification methods. Chapter 12 proposes to partition networks into unconnected or overlapping communities. Finally, Chapter 13 concludes by describing how to partition very large networks into communities.

Further information that could not be included in this book is available at the book's Web address1. In particular, as this book is being printed in black and white, some figures will have lost their clarity when compared to their original color version. This is especially true for images in Chapters 9 and , respectively, about airspace partitioning and image segmentation. To overcome this problem, the original color versions of the figures concerned are also available at the book's Web address. You will also find at this Web address different graphs used in this book, as well as links to several softwares for graph partitioning.

Charles-Edmond BICHOT and Patrick SIARRY
August 2011

* * *

1 The book's Website, "Graph partitioning: optimization and applications", is available at: perso.ec-lyon.fr/charles-edmond.bichot/.

# Chapter 1

# General Introduction to Graph Partitioning 1

# 1.1. Partitioning

The word "partitioning" represents the action of creating a partition by segmenting a set into several parts. In mathematics, a partition of a set _X_ is a family of pairwise disjoint parts of _X_ whose union is the set _X_ . Therefore, each element of _X_ belongs to one, and only one of the parts of the partition.

Partitioning is used to solve a large number of engineering problems. There are many examples of partitioning applications: data mining, VLSI design, load balancing, parallel computing, matrix computation, fluid dynamics, image analysis, air traffic control, etc.

Creating a partition consists of distributing a set of objects into several subsets. In order to distribute the objects in these different subsets, it may be useful to compare them. Thus, each object will be associated with a few other objects. When each object is associated with all other objects, then the grouping is known as clustering, and not partitioning. The resulting associations would be quantified. If the set of all these objects is finite, then all the conditions would be met to create a graph from these objects, in other words to materialize the associations between objects. The next step will be to build a partition of this graph. Partitioning a set of objects generally aims to distribute these objects into parts having strong internal links and weak links between parts. That is known as the objective of partitioning or objective function. This objective function varies according to the specific problem which is to be solved. We emphasize the fact that in this book we mainly focus on the optimization problem of minimizing an objective function. Thus, we will generally seek to minimize the links between parts rather than maximize them.

The problem of graph partitioning optimization is common to several disciplines:

– it is part of graph theory problems and therefore discrete mathematics. Discrete mathematics relates to discrete structures, i.e. finite or countable sets;

– it is also part of the combinatorial optimization problems. A combinatorial optimization problem tries to find out the best solution within a set of discrete solutions;

– it is solved using computing.

Graph partitioning is, therefore, a discipline situated between computer science and applied mathematics.

Because of the diversity of its applications, the graph partitioning optimization problem evolves in a multitude of problems. However, we can group them into two large categories. The nature of a graph partitioning problem is very different according to whether we seek to obtain parts of very similar sizes or without any size constraint. This observation leads us to distinguish between two different graph partitioning problems:

– _constrained partitioning_ , when the parts of the partition are of similar sizes;

– _unconstrained partitioning_ , when the parts can be of (very) different sizes.

Before giving a precise definition of these two problems, this introductory chapter presents a few reminders on mathematics in section 1.2 and graph theory in section 1.3. These will be followed by a formal description of the graph partitioning optimization problem in section 1.4 and its different objectives in section 1.5. Then the two main categories of graph partitioning problems will be presented: first the constrained partitioning problem in section 1.6, and then the unconstrained partitioning problem in section 1.7. Finally, section 1.10 will study the NP-hardness of several graph partitioning problems.

# 1.2. Mathematical notions

DEFINITION 1.1. (CARDINALITY OF A SET)– _Let X be a finite set of n elements. The cardinality of the set X is the number n of elements of X , and is denoted by_ | _X_ |. _Therefore, we have_ | _X_ | = _n_.

In mathematics, the partition of a set V of objects is defined as follows:

DEFINITION 1.2. (PARTITION)– _Let V be an ordinary set. A set P of subsets of V is called a partition of V if:_

– _no element of P is empty;_

– _the elements of P are pairwise disjoint;_

– _the union of the elements of P is equal to V._

_The elements of P are called the parts of the partition P. The cardinality of the partition P is therefore the number of parts of P. In graph partitioning, the number of parts of P is often denoted k._

It is a bad habit to use the word partition instead of part, which creates ambiguity. Thus, instead of saying "the partition _P_ 1 of _P_ ", it is better to say "the part _P_ 1 of the partition _P_ ".

As previously mentioned, the graph partitioning problem is a combinatorial optimization problem, which is a branch of discrete mathematics. Discrete mathematics represent the study of mathematical structures, where the notion of continuity doesn't exist. Thus, sets studied in discrete mathematics are countable, and therefore countable sets are sometimes called discrete sets.

We can now give a formal definition of a combinatorial optimization problem:

DEFINITION 1.3. (COMBINATORIAL OPTIMIZATION PROBLEM)– _A combinatorial optimization problem is defined from a triplet_ ( _S, p, f_ ) _such that_ :

– _S is a discrete set called research space (also called solution space);_

– _p is a predicate on S, i.e. a function of S in_ { _true, false_ };

– _f : S_ → _R associates with each element x_ ∈ _S a cost f_ ( _x_ ).

T _he function f is called the objective function or the cost function of the problem. The predicate p creates a set S a_ = { _x_ ∈ _S such that P_ ( _x_ ) _is true_ } _which is called the set of admissible solutions of the problem. Each element of S is called a solution of the problem, and each element of S a an admissible solution of the problem._

_A combinatorial optimization problem aims to find the element_   ∈ _S a (or the subset of elements of Sa) that minimizes f:_

[1.1]

_Note_ : The combinatorial optimization problem which looks for a maximum element instead of a minimum element, is of the same nature because:

[1.2]

Combinatorial optimization problems are generally very expensive to solve optimally. This is particularly the case of graph partitioning problems, which are often NP-hard, as we will see in section 1.10.

The research space _S_ can be described as the set of solutions of the combinatorial optimization problem, which we seek to solve without taking into account the problem's constraints. For example, let's consider the combinatorial optimization problem aiming to find a partition of the vertices of a graph _G_ = ( _V_ , _E_ ) into _k_ parts of equal sizes (we choose _k_ divisor of | _V_ |). This problem will have as research space S _t_ he set of all possible partitions of _V_ , i.e. partitions of parts of any sizes whose number of parts ranges from one to | _V_ |. However, the set of admissible solutions to problem Sa must take the constraints into account. Thus, in our example, the set Sa will be made of partitions of V into k parts of equal sizes.

DEFINITION 1.4. (GLOBAL OPTIMUM, LOCAL OPTIMUM)– _Let_ ( _S, p, f_ ) _be acombinatorial optimization problem and Sa the set of admissible solutions of the problem resulting from the predicate p. Let  ∈ Sa._

– _If we can prove that_ ∀ _x_ ∈ _S a, f _( ) ≤ _f_ ( _x_ ) _, then_  _is called a global optimum (minimum) of the problem;_

– _If there is a set of admissible solutions X_ ⊂ _S a_, | _X_ | ≥ 2, _and_   ∈ _X such that ∀x_ ∈ _X_ , _f_ ( ) ≤ _f_ ( _x_ ), _then_   _is called a local optimum (minimum) of the problem._

The research space _S_ has a _topology_. It is very useful to know the characteristics of the topology of S in order to understand the purpose of the metaheuristics mechanisms and to improve them (Chapters 6 and  present how to adapt metaheuristics to graph partitioning). This topology results from the notion of proximity between two solutions, also called configurations. The distance between two configurations represents the minimum number of elementary modifications necessary to go from one to another. Besides, because a value _f_ ( _x_ ) is associated with each configuration _x_ , the solution space is characterized by a multi-dimensional chart called _energetic landscape_ 1. In this energetic landscape, the local or global optima form energetic wells around them; these _energetic wells_ are also named basins of attraction. Before saying that an admissible solution   ∈ _X_ (see definition 1.4) of the problem is a local minimum, we must generally check that the size of the set _X_ is large enough in relation to the size of _S a_. Figure 1.1 represents the continuous equivalent of the energetic landscape of a cost function in a single-dimension solution space.

# 1.3. Graphs

This section is a reminder of some definitions of the graph theory.

DEFINITION 1.5. (GRAPH)– _Let V be a non-empty set of n V elements and E a set of nE pairs of elements of V . A graph G is the pair (V , E). The elements of V are called_

**Figure 1.1.** _Continuous equivalent of the energetic landscape of a cost function f (x) in a single-dimension research space_

_vertices of the graph and those of E the arcs or edges of the graph, depending on whether they are directed or not._

An edge represents a connection between exactly two vertices. When there is a connection between at least three vertices, these vertices are connected by what is called a hyperedge. By extension, we no can longer use the term graph, but hypergraph.

DEFINITION 1.6. (HYPERGRAPH)– _Let V be a non-empty set of elements and E a set of non-empty subsets of elements of V . A hypergraph G is the pair (V, E). The elements of E are called hyperedges._

_Note_ : The problem of hypergraph partitioning is presented in Chapter 3.

When the associations between vertices are not reciprocal, for example, if the vertex _v_ 1 is connected with the vertex _v_ 2, without the vertex _v_ 2 being connected with _v_ 1, then the edges are said to be directed and are called arcs.

DEFINITION 1.7. (DIRECTED AND UNDIRECTED GRAPHS)– _Let G = (V, E) be a graph._

_If for each edge_ ( _v_ 1, _v_ 2) ∈ _E, we have_ ( _v_ 2, _v_ 1) ∈ _E, then the graph is undirected and the elements of E are called edges of the graph. In that case, an edge is equally denoted by:_ ( _v_ 1, _v_ 2) = ( _v_ 2, _v_ 1) = _e_ ∈ _E with v_ 1 _and v_ 2 _in V ._

_In the opposite case, the graph is said to be directed and the elements of E are called arcs of the graph._

DEFINITION 1.8. (ADJACENT VERTICES AND EDGES, INCIDENT EDGES)– _Let G_ = ( _V, E_ ) _be an undirected graph._

_Let e_ = ( _v_ 1, _v_ 2) ∈ _E be an edge. The vertices v_ 1 _and v_ 2 _are said to be the adjacent vertices of the edge e. Similarly, e is said to be incident to the vertices v_ 1 _and v_ 2.

_Let e_ 1 _and e_ 2 _be two edges. e_ 1 _and e_ 2 _are said to be adjacent if they are incident to the same vertex, i.e. e_ 1 = ( _v, v_ 1) _and e_ 2 = ( _v, v_ 2).

DEFINITION 1.9. (DEGREE OF A VERTEX)– _In an undirected graph G_ = ( _V, E_ ), _the degree of a vertex v_ ∈ _V is the number of edges incident to the vertex :_

[1.3]

DEFINITION 1.10. (LOOP AND MULTIPLE EDGES)– A _n edge is called a loop if both of its extremities are identical._

_If two or more edges are incident to the same two vertices, then the edge is said to be multiple and these two edges are parallel. In that case, the multiplicity of an edge is the number of its parallel edges, including itself._

DEFINITION 1.11. (SIMPLE GRAPH)– _An undirected graph is said to be simple if it has neither loop nor multiple edges._

DEFINITION 1.12. (REGULAR GRAPH)– _An undirected graph is said to be regular when all its vertices have the same number of neighbors._

When the associations between vertices are quantified, the edges of the graph have a weight, and the graph is said to be weighted.

DEFINITION 1.13. (WEIGHTED GRAPH)– _Let G_ = ( _V_ , _E_ ) _be a graph. The graph is said to be weighted if a real value, w_ ( _e_ ), _is associated with each edge e_ ∈ _E. The value w_ ( _e_ ) _is called the weight of e. By extension, a pair of vertices_ ( _v_ 1, _v_ 2) ∈ _V_ 2 _such that_ ( _v_ 1, _v_ 2) ∉ _E is considered to have a zero weight: w_ ( _v_ 1, _v_ 2) = 0.

_The weight of a subset X of edges of E is the sum of the weights of the elements of X : w_ ( _X_ ) = ∑ _e_ ∈ _X_ _w_ ( _e_ ).

Likewise, we often associate with the vertices _v_ ∈ _V_ of the graph to a strictly positive integer called the weight of v and denoted by _w_ ( _v_ ).

_Note_ : In the case of graph partitioning, the weight of an edge or vertex is most often a strictly positive integer.

We must be careful not to confuse the cardinality of a set of vertices (respectively edges) and the weight of a set of vertices (respectively edges). They are equal if the vertices (respectively edges) are not weighted.

In the case of weighted graphs, we redefine the degree of a vertex _v_ ∈ _V_ as the sum of the weights of the edges adjacent to this vertex. Thus:

[1.4]

For an unweighted graph, we can associate with each edge, and each vertex a unit weight.

For any graph _G_ = ( _V_ , _E_ ), we can define a bijection _u_ : _V_ → {1, . . . , _n V_} which associates an integer with every element of _V_ . The function _u_ can count the elements of _V_ . For convenience, with each element of the set of vertices _V_ of a graph _G_ , we will associate a rank in _V_ , and we will merge the vertex _v i_ of _V_ with its rank _i_ in _V_ . For example, for each ( _v i_, _v j_ ) ∈ _E_ , _w_ ( _v i_, _v j_ ) will be denoted by _w_ ( _i_ , _j_ ), and _w_ ( _v i_) by _w_ ( _i_ ).

For any graph, we can define a unique function, _weight: E_ →  , which associates each edge with its weight. This _weight_ function can be represented as a matrix. The adjacency matrix of a graph _G_ can be defined as the matrix that associates each edge of _G_ with its weight.

DEFINITION 1.14. (ADJACENCY MATRIX)– _Let G_ = ( _V, E_ ) _be a simple graph. The matrix M Adj has dimension nV_ × _n V , such that_ ∀( _i, j_ ) ∈ {1, . . . , _n V_}2:

[[1.5]

_is called the_ adjacency matrix _of the graph G_.

DEFINITION 1.15. (DEGREE MATRIX)– _Let G_ = ( _V, E_ ) _be a simple graph. The matrix M Deg has dimension nV_ × _n V , such that_ ∀( _i, j_ ) ∈ {1, . . . , _n V_}2:

[[1.6]

_is called the degree matrix of the graph G._

DEFINITION 1.16. (LAPLACIAN MATRIX)– _Let G_ = ( _V, E_ ) _be a simple graph. The matrix_

[[1.7]

_is called the Laplacian matrix of G._

DEFINITION 1.17. (PATH IN A GRAPH)– _Let G_ = ( _V, E_ ) _be a graph and_ ( _v_ 1, _v_ 2) _be two distinct vertices. If a succession of edges_ ( _or correctly directed arcs in the case of a directed graph_ ) _are allowed to reach v_ 2 _from v_ 1, _then we can say that there is a_ path _from v_ 1 _to v_ 2.

DEFINITION 1.18. (CONNECTED GRAPH)– _Let G_ = ( _V, E_ ) _be a graph. This graph is said to be_ connected _if for any pair of vertices v_ 1 _and v_ 2 _a path from v_ 1 _to v_ 2 _exists_.

The matching of a graph is a subset of edges of this graph which have no incident vertex in common.

**Figure 1.2.** _Difference between a maximal matching and a maximum matching_

DEFINITION 1.19. (MATCHING)– _Let G_ = ( _V, E_ ) _be a graph. The matching M of the graph G is a set of pairwise non-adjacent edges._

_A matching M is maximal if every edge of the graph is incident with at least one edge of M._

_A matching is maximum when it contains the highest possible number of edges._

_Note_ : Every _maximum_ matching is also a _maximal_ matching. Let _M_ be a _maximum_ matching and assume that there is a set of edges _M_ ′ of _E_ , not in _M_ , such that _M_ ∪ _M_ ′ is a _maximal_ matching. Then _M_ is not a _maximum_ matching, as there is a matching, _M_ ∪ _M_ ′ that contains more edges. That is absurd, therefore _M_ is a _maximal_ matching. The reciprocal is not necessarily true: in a linear graph with four vertices, the matching made up of the edge formed by the two central vertices is _maximal_ but not _maximum_ (see Figure 1.2). A graph can have several _maximum_ matchings (and a _fortiori_ several _maximal_ matchings).

# 1.4. Formal description of the graph partitioning problem

Before presenting the general problem of graph partitioning, we have to define the partition of a graph. As mentioned in the previous section, a graph is a pair formed by a set of vertices and a set of edges. It is therefore possible to partition, in a mathematical sense, the set of vertices as well as the set of edges. However, although some problems seek to partition the edges of a graph [HOL 81], graph partitioning is mostly understood as the partition of the vertices of the graph.

DEFINITION 1.20. (PARTITION OF THE VERTICES OF A GRAPH)– _Let G_ = ( _V, E_ ) _be a graph and P k_ = { _V 1_, . . . , _V k_} _a set of k subsets of V . P k_ _is said to be a partition of G if:_

– _no element of P k is empty:_

∀ _i_ ∈ {1, . . . , _k_ }, _S i_ ≠ ∅;

∀ _i_ ∈ {1, . . . , _k_ }, _S i_ ≠ ∅;

– _the elements of P k are pairwise disjoint:_

∀( _i, j_ ) ∈ {1, . . . , _k_ }2, _i_ ≠ _j_ , _V i_ ∩ _V j_ = ∅;

– _the union of all the elements of P k is equal to V :_

_The elements V i of Pk are called the parts of the partition. The number k is called the cardinality of the partition, or the number of parts of the partition._

The definition of a partition of the vertices of a graph does not restrict itself to the size of the parts. To remedy this and measure the size of the parts of a partition, the concept of a partition _balance_ is introduced. The term balance refers to the _measure of the weight_.

DEFINITION 1.21. (BALANCE OF A PARTITION)– _Let G_ = ( _V, E_ ) _be a graph and P k_ = { _V_ 1, . . . , _V_ k } _a partition of this graph into k parts. The average weight of a part V i of Pk is_

where   ∈   _is the first integer which is greater than or equal to x._

_The balance bal(P k ) of the partition Pk is the division of Pk 's maximal weighted element by the average weight of a part:_

[[1.8]

Thus, a partition whose partitioning balance is unitary has parts of identical weights (rounded by one). For example, the bisection of a graph of size (2 _n_ \+ 1) will have a balance of 1.00, if and only if the largest part is of size _n_ \+ 1 and the other part is of size _n_.

As we have previously seen, the graph partitioning problem is a combinatorial optimization problem. To present it simply, the general graph partitioning problem consists in finding a partition of a graph which satisfies a predicate _p_ and minimizes an objective function _f_ . Here a more formal definition is provided:

DEFINITION 1.22. (GENERAL GRAPH _k_ -PARTITIONING OPTIMIZATION PROBLEM)– _Let G = (V, E) be a graph and let k_ ≥ 2 _be an integer. We can define the triplet (S, p, f ) that characterizes the general graph k-partitioning optimization problem such that:_

– _the research space S is defined as the set of all possible partitions of V (these partitions have a cardinality that ranges from 1 to_ | _V_ |;

– _let p′ be a predicate defined by P_ ∈ _V_ , _p′_ _(P) = true if and only if_ | _P_ | = _k. The predicate p of the general problem is defined in S such that p(P) = true_ ⇒ _p′_ ( _P_ ) = _true, thus it can be more restrictive than p′_ ;

– _let f : S_ →  _be an objective function._

_The set of admissible solutions of the problem is defined by:_

_The general graph partitioning problem consists in finding the partition  k ∈ Sa that minimizes f:_

[1.9]

It can be noticed that _S a_ ⊆ { _P_ | ∈ _S_ _such that_ | _P_ | = _k_ }. This means that an admissible solution of a graph partitioning optimization problem can be more restrictive than merely satisfying the predicate _p′_ (i.e. | _P_ | = _k_ ). That is why we have introduced the predicate p and not just the predicate _p′_.

By definition, the set of vertices of the graph is finite. The number of partitions of a set of _n_ elements is called the Bell number relative to _n_ , and is denoted by _B n_. It represents the number of distinct equivalence relations on a set of _n_ elements. _B_ 0 = 1. The Bell number satisfies the following recursive equation:

[1.10]

where   is the binomial coefficient of n and k, i.e. the number of parts of k elements from a set of _n_ elements.

The number of partitions into _k_ parts of a set of _n_ elements is called the Stirling number of the second kind. This number is denoted by _S n,k_ and its value is:

[1.11]

There is a relation between the Bell number and the Stirling number of the second kind:

[1.12]

Thus, because the set of vertices is finite, the set of solutions S is a finite set. It is therefore a discrete set, according to the definition of a combinatorial optimization problem (see section 1.2).

The general graph partitioning optimization problem is defined by using a pair formed by a predicate and an objective function: ( _p, f_ ). Thus, graph partitioning can be divided into numerous problems, depending on the nature of _p_ and _f_ . However, these problems can be classified into two main different categories, depending on the nature of the predicate _p_ : the constrained graph partitioning problem introduced in section 1.6 and the unconstrained graph partitioning problem described in section 1.7.

# 1.5. Objective functions for graph partitioning

The different objective functions for graph partitioning are based on two concepts: inter-class disparities and intra-class similarities. The cut between parts often represents inter-class disparities and the weight of parts is used to characterize intra-class similarities.

DEFINITION 1.23. (CUT FUNCTION)– _Let G_ = ( _V, E_ ) _be a graph. Let V a and Vb be two subsets of V , and we can define the cut between these two subsets as:_

[1.13]

For the rest of this section, let _P k_ = { _V_ 1, . . . , _V k_} be a partition of _V_ into _k_ parts. The objective functions presented will be defined with regard to this partition.

The simplest of the objective functions used in graph partitioning is called the cut of a partition. It aims to minimize the sum of the weight of the edges between parts of the partition _P k_ . It is already used by B. Kernighan and S. Lin [KER 70]:

DEFINITION 1.24. (CUT OBJECTIVE FUNCTION)

[1.14]

The objective function that aims to minimize, for each part, the ratio between its cut and its weight, is called ratio cut. It was introduced by Y.-C. Wei and C.-K. Cheng [WEI 89]:

DEFINITION 1.25. (RATIO CUT OBJECTIVE FUNCTION)

[1.15]

However, when the term ratio cut is used, we have to be careful that other objective functions are also called by the same name in the literature [WAN 03].

The third and last function widely used in graph partitioning is called normalized cut. This function, which is the most recent, was presented by J. Chi and J. Malik [SHI 00]. It aims to minimize, for each part, the ratio between its cut and the sum of the weight of the edges adjacent to at least one of its vertices. In other words, it aims to minimize, for each part, the ratio between the sum of the weight of the edges adjacent to exactly one of its vertices and the sum of the weight of the edges adjacent to at least one of its vertices:

DEFINITION 1.26. (NORMALIZED CUT OBJECTIVE FUNCTION)

[1.16]

The normalized cut function can isolate regions of the graph where the vertices are strongly inter-connected. Its authors created it to replace the ratio cut in their image segmentation problem.

_Note_ : In the case of graphs weighted by strictly positive weights, which is almost always the case in graph partitioning problems, all the objective functions presented before are positive.

All these objective functions can define the _energy_ of a part of a partition. The term energy is used in reference to the concept of energetic landscape presented in section 1.2. This energy is the value brought by the part of the partition to the objective function. In the case of the cut objective function, the energy of a part _V i_ will be:

[1.17]

In the case of the ratio cut, the energy of a part _V i_ will be:

[1.18]

In the case of the normalized cut, the energy of a part _V i_ will be:

[1.19]

# 1.6. Constrained graph partitioning

Classic literature on graph partitioning focuses on the constrained graph partitioning optimization problem. Most graph partitioning tools solve this problem, such as METIS, JOSTLE, CHACO, SCOTCH or PARTY, as we will see in Chapter 2 and in the Appendix.

One of the first chapters to present this problem was from B. Kernighan and S. Lin [KER 70]. The constrained graph partitioning problem is often used to illustrate new graph partitioning methods. Indeed, in the chapter introducing the spectral method [POT 90], A. Pothen, H. Simon and K.-P. Liou applied this new method to this problem. Moreover, the chapters of S. Barnard and H. Simon introducing the multilevel method [BAR 93, BAR 94] again applied this new method to a constrained graph partitioning problem.

Constrained graph partitioning is mainly used to solve different engineering problems like high performance computing, linear systems resolution, meshing, and VLSI circuits conception. However, criticism has been raised regarding the use of this approach to model these problems, more particularly the case for [HEN 98].

In literature, constrained graph partitioning is often called _multi-way graph partitioning_ [YAR 00] or _k-way graph partitioning_ [KAR 98].

The problem of constrained graph partitioning consists of finding a partition into _k_ parts, which should imperatively respect a strict balance while minimizing an objective function _f_.

The objective function, minimized by constrained partitioning, is most often the _cut_ objective function (see definition 1.24).

Let's consider the other objective functions. If a partition _P k_ has parts of identical weights, then its ratio cut (section 1.5) is equal to its cut, up to a multiplicative constant. In fact, in that case:

[1.20]

Thus, for the problem of constrained graph partitioning, minimizing the ratio cut is equivalent to minimizing the cut.

Besides, when considering a partition _P k_ whose parts have identical weight, the use of the normalized cut loses its meaning. Indeed, this cost function has been created to isolate from other regions those whose vertices are connected, which is often contradictory to the search for equal parts. Therefore, to the best of our knowledge, this cost function is never used in problems of constrained partitioning.

DEFINITION 1.27. (CONSTRAINED GRAPH PARTITIONING)– _Let G_ = ( _V, E_ ) _be a graph, k a number of parts, and bal_max a maximal partitioning balance. Let S be the solution space, defined by the general graph partitioning problem. The set S a of admissible solutions of the problem is defined by:_

_The_ constrained graph partitioning _problem consists of finding_   ∈ _S a_ _that minimizes the cost function f:_

[1.21]

_Notes:_

– the maximal partitioning balance of constrained graph partitioning problems is often very small: _bal_max_ ∈ [1, 0; 1, 05];

– within constrained graph partitioning problems, a small relaxation of the partitioning balance is often accepted. Indeed, with a value of the balance slightly greater than the unit, partitions of lower cut can be found, and such solutions are often technically not detrimental to the resolution of the problem [SIM 97].

# 1.7. Unconstrained graph partitioning

The unconstrained graph partitioning optimization problem is slightly more exotic than the constrained graph partitioning optimization problem. We used the term exotic because this problem is divided into very similar problems and its objective functions have been regularly improved, or at least modified. Finally the main aim of constrained graph partitioning consists of finding a partition respecting a restrictive balance, the main aim of unconstrained graph partitioning is to minimize an objective function. Moreover, if there is no other aim for unconstrained graph partitioning, there is a second one for constrained graph partitioning, which is to minimize the cost function.

Unconstrained graph partitioning is quite similar, in its formulation, to data partitioning or _clustering_. The data partitioning problem consists of merging similar elements into sets that are as distinct as possible. To compare two elements, the notion of distance is used. That brings the data partitioning problem down to the creation of subsets of elements as distinct from each other as possible. And yet, we can easily create a graph whose vertices are these elements and whose edges have for weight the inverse of the distance between two elements. In the case where the number of parts is fixed, a data partitioning problem is similar to an unconstrained graph partitioning problem. As a result, many studies have sought to solve data partitioning problems using graph partitioning tools. However, graphs modeled in this way are often complete, because there is a distance for each pair of elements, and so, all the vertices of the graph will be connected.

Unconstrained graph partitioning is widely used not only in image segmentationSHI 98b, SHI 98a, PER 98, MEI 00, SHI 00, GDA 01, WAN 01, MAR 04, BEN 05], but also in texts classification [DHI 01, ZHA 01, DIN 01, ZHA 04, BIC 09]. In the case of image segmentation, numerous chapters compare different objective functions and suggest new functions including new criteria [SHI 98b, SHI 00, PER 98, GDA 01, WAN 01, FEL 04]. Several chapters solve VLSI circuit partitioning problems by using an unconstrained graph partitioning approach [HAG 92, RIE 94, ALP 95]. However, in that case, VLSI circuits are more often considered as hypergraphs than graphs [KER 70]. Other more unusual problems, such as airspace partitioning (see [Chapter 9), are also unconstrained graph partitioning problems.

The objective functions that an unconstrained graph partitioning problem seeks to minimize are various. Section 1.5 presents the ratio cut and the normalized cut. These two objective functions, with several similar variants, are those that occur the most within unconstrained graph partitioning problems. In some cases, and to some extent, we can consider, for unconstrained graph partitioning, that the constraint on the partitioning balance is integrated with the objective function. Thus, unlike the constrained case, most problems omit the constraint on the partitioning balance and only enunciate the objective function to minimize. The constraint on the partition balance will therefore not be included in the definition of unconstrained graph partitioning, but may be added afterward:

DEFINITION 1.28. (UNCONSTRAINED GRAPH PARTITIONING)– _Let G_ = ( _V, E_ ) _be a graph and k a number of parts. Let S be the solution space, defined by the general graph partitioning problem. The set S a of admissible solutions of the problem is defined by:_

_The problem of constrained graph partitioning consists of finding a partition_   ∈ _S a that minimizes the cost function f (as the normalized cut or the ratio cut):_

[1.22]

As the set of admissible solutions in the constrained case is much smaller than in the unconstrained case, finding a partition that satisfies the latter is easier. In constraint case, an algorithm that deviates from the required balance will struggle to find a partition that verifies it afterward. It is therefore difficult to start a search outside the admissible solution space in order to get back to it. Thus, it may seem easier to find a good solution with a smooth balance than a strong one. However, this is not the case. Indeed, if it is easier to find an admissible solution in the unconstrained case, this is because the admissible solution space is bigger. This does not mean that the admissible solution space is easier to explore, and that it is easier to seek a good solution. Besides, a local minimum in the constrained case is not necessarily close to a local minimum in the unconstrained case.

To a certain extent and when a maximal balance is imposed, unconstrained graph partitioning can be considered as a multiobjective optimization problem that consists of minimizing one of the objective functions of section 1.5 and minimizing the balance of the partition [RUM 02, GIL 06, SEL 03, SEL 06]. To overcome the shortcomings of the classic objective functions and to integrate other partially conflicting objectives, other chapters have considered the graph partitioning problem as a multiobjective problem [SCH 99].

# 1.8. Differences between constrained and unconstrained partitioning

In the previous sections, we have presented the two main graph partitioning optimization problem categories: the constrained graph partitioning problem and the unconstrained graph partitioning problem. However, although these two problems have a similar formulation, algorithms used to solve them are generally quite different, if not in their principles then in their implementations and purpose. In fact, at the risk of oversimplifying, these two problems primarily differ in their priorities:

– the first purpose of the **constrained graph partitioning** problem is to find a partition with parts of identical weights. Its second purpose is to reduce the cut of the partition;

– the **unconstrained graph partitioning** problem aims to maximize the cut inside parts while minimizing the cut between parts. The principle adopted is to avoid extremely different parts sizes, except in the case of an extremely low cut between parts. Thus, the first purpose of unconstrained graph partitioning is to minimize the cut, and a second, but not obligatory purpose, is to balance the parts.

If we take an efficient algorithm designed for one of these two problems and then we use it to solve the other problem, then we are very likely to get poor results. Indeed, reversing the order of priorities between these two problems leads to poor results [BIC 06, BIC 07].

To the best of our knowledge, we deplore the absence of chapters in the literature, which addresses and exhaustively compares these two problems. However, unconstrained graph partitioning problems are numerous and efforts have been made by Inderjit S. Dhillon to synthesize them [DHI 04, DHI 07].

# 1.9. From bisection to _k_ -partitioning: he recursive bisection method

Many graph partitioning algorithms only focus on the case of graph bisection [RON 05, MAR 05, MAR 06, CHA 07]. In this section we will present two ways of creating _k_ -partitions from a graph bisection algorithm. The method used will depend on the characteristics of the bisection algorithm.

When it is possible to precisely adjust the partitioning balance, the recursive bisection technique can adapt a bisection algorithm to k-partitioning. However, when it is not possible to adjust the partitioning balance, the recursive bisection technique can only create 2 _l_ -partitions (with _l_ an integer), where the partitions with the number of parts are a power of 2.

We draw the reader's attention to the fact that, despite all the interest in being able to adapt bisection methods to _k_ -partitioning, it was shown that the recursive bisection method could not explore, and therefore find, some solutions to the _k_ -partitioning problem [SIM 97]. These solutions can only be found using direct _k_ -partitioning methods, hence their interest. However, in practice, recursive bisection methods are generally at least as efficient as the direct _k_ -partitioning methods.

Initially we will study the case of a recursive bisection algorithm that can create 2 _l_ -partitions from an ordinary bisection algorithm. Secondly, we will study the case of a bisection algorithm that can create a bisection that is strongly unbalanced, which means that both parts have significantly different weights. We will see that this bisection algorithm can be used recursively to create a _k_ -partition.

## 1.9.1. _Creating a partition with a number of parts a power of 2, from a graph bisection algorithm_

In this section, we present a simple and efficient algorithm that allows us to create a 2 _l_ -partition (with _l_ an integer), from an ordinary bisection algorithm. In practice, only strongly balanced bisection algorithms are used within this algorithm, those allowing a configurable partition balance rather use the algorithm presented in the next section.

Algorithm 1.1 describes the method of adapting a _bisection_ algorithm named bisection to 2 _l_ -partitioning. It uses the recursive function Iterate that takes as parameters the part _V′_ of the graph to partition, the number of parts _k′_ into which we will partition _V′_ , and the rank num of the first part of the partition of _V′_ in the final partition _P k_ .

## 1.9.2. _Creating a k-partition from a graph bisection algorithm using the partitioning balance_

In this section, we present a recursive bisection algorithm that allows us to create a _k_ -partition from a bisection algorithm with a configurable partition balance, i.e. from

* * *

**Algorithm 1.1.** Recursive bisection algorithm allowing to create a partition into 2 _l_ parts

* * *

* * *

* * *

**Algorithm 1.2.** Recursive bisection algorithm allowing to create a _k_ -partition

* * *

* * *

a bisection algorithm that returns a partition where both parts have their weight given as a parameter of algorithm. Thus, the bisection algorithm named _bisection_ , called in Algorithm 1.2, takes two parameters: the part of the graph to partition and the weight of the first part (the weight of the second part is therefore the remaining weight).

Algorithm 1.2 describes the method of adapting a bisection algorithm named _bisection_ to _k_ -partitioning. It uses the recursive function ITERATE that takes as parameters the part _V′_ of the graph to partition, the number of parts _k′_ into which we will partition _V′_ , and the rank _num_ of the first part of the partition of _V′_ in the final partition _P k_ . The value of   is the greater integer lower than   and is the number of parts in which the partition of   has to be made. The function bisection must find a bisection of _V′_ in two parts   and   such that:

[1.23]

# 1.10. NP-hardness of graph partitioning optimization problems

## 1.10.1. _The case of constrained graph partitioning_

To demonstrate that an optimization problem is NP-hard, we can show that the related decision problem is NP-complete. The decision problem associated with the constrained graph partitioning problem, as presented in section 1.6, can be formalized as follows:

**Decision problem associated with the constrained graph partitioning problem:**

**Problem's instance.** Let _G_ = ( _V, E_ ) be a graph weighted on its vertices by ∀ _v_ ∈ _V_ , _w_ ( _v_ ) ∈  ∗ and on its edges by ∀ _e_ ∈ _E_ , _w_ ( _e_ ) ∈  ∗. We have _weight max_ ∈  ∗ and _cut max_ ∈  ∗.

**Question.** Is there a partition _P k_ of _V_ in _k_ unconnected parts _V_ 1, . . . , _V k_ such that for each part _V i_,

and such that:

**Solution.** This problem is NP-complete for _weight max_ ≥ 3, even when the weights of the vertices and the edges are the same. It can be solved in polynomial time when _weight max_ = 2.

This decision problem, called _graph partitioning_ , has been processed in [HYA 73] and as a reference in [GAR 79]. Thus, the constrained graph partitioning problem is NP-hard.

## 1.10.2. _The case of unconstrained graph partitioning_

The problem of unconstrained graph partitioning is defined in section 1.7. The aim of this subsection is to show that the problem is NP-hard for the objective functions of normalized cut and ratio cut.

To demonstrate that an optimization problem is NP-hard, we can show that the related decision problem, denoted by Π, is NP-complete. To demonstrate that Π is NP-complete, according to [GAR 79], we can follow these three steps:

1) show that Π is in NP;

2) find a NP-complete problem Π′ close to Π;

3) reduce Π′ to Π: find a subproblem of Π′ equivalent to Π.

### 1.10.2.1. _NP-hardness of the normalized cut_

The NP-hardness of the unconstrained graph partitioning optimization problem for the normalized cut has been shown in [SHI 00] by reducing it to the partition problem. The NP-complete decision problem associated with the partition problem is enunciated as follows:

**Decision problem associated with the partition problem:**

**Problem's instance.** Let's consider a finite set _V_ and a weight function for all the elements _v_ ∈ _V_ : _w_ ( _v_ ) ∈  ∗.

**Question.** Is there a subset _V′_ ⊆ _V_ such that

The demonstration of the NP-hardness of the partitioning problem by using the normalized cut is based on the construction of a very specific graph. The property of this graph is that a bisection of this graph has a normalized cut which is sufficiently small if and only if we can find a subset of _V_ with the sum of the weights equal to half the weight of _V_ . The decision problem of the normalized cut therefore comes down to the decision problem of the partition, thus concluding the demonstration.

### 1.10.2.2. _NP-hardness of the ratio cut_

The decision problem Π associated with the ratio cut optimization problem can be formalized as follows:

**Decision problem associated with the ratio cut optimization problem:**

**Problem's instance.** Let _G_ = ( _V, E_ ) be a graph weighted on the vertices by ∀ _v_ ∈ _V_ , _w_ ( _v_ ) ∈  ∗ and on the edges by ∀ _e_ ∈ _E_ , _w_ ( _E_ ) ∈  ∗. We have _cut max_ ∈  ∗.

**Question.** Is there a partition _P k_ of _V_ in k unconnected subsets _V_ 1, . . . , _V_ k , non-empty, such that _ratio_ ( _P k_) ≤ _cut max_?

To demonstrate that Π is in NP, all we have to do is show that the verification of a solution to the problem Π is executed in polynomial time. Let _P k_ be a solution to the problem Π. The verification that the sets _V i_ ∈ _P k_ are unconnected and lower than a maximal weight is performed in _O_ (| _V_ |). The calculation of the ratio cut is performed by covering all the edges, i.e. in _O_ (| _E_ |). The verification that each set _V i_ is non-empty is an operation in _O_ ( _k_ ). Thus, the verification that a partition is a solution of Π is done in polynomial time.

There are several methods to prove that an optimization problem is NP-hard [GAR 79]. The most common method consists of restricting the associated decision problem to show that a particular case of it is a NP-complete problem.

The problem of the minimum cut into finite sets, called _minimum cut into bounded sets_ [GAR 79], is similar to the problem Π. It is formulated as follows:

**Decision problem associated with the minimum cut problem:**

**Problem's instance.** Let _G_ = ( _V, E_ ) be a graph weighted on the vertices by ∀ _v_ ∈ _V_ , _w_ ( _v_ ) ∈  + and on the edges by ∀ _e_ ∈ _E_ , _w_ ( _e_ ) ∈  +. Let _v_ 1 and _v_ 2 be two distinct vertices of _V_. Let's consider _weight max_ ≤ _w_ ( _V_ ) and _cut max_ as two positive integers.

**Question.** Is there a bisection _P_ 2 of V into two disjoint subsets _V_ 1, _V_ 2, that verifies _v_ 1 ∈ _V_ 1, _v_ 2 ∈ _V_ 2, _w_ ( _V_ 1) ≤ _weight max_ and _w_ ( _V_ 2) ≤ _weight max_, such that _cut_ ( _V_ 1, _V_ 2) ≤ _cut max_?

Let's now show that Π is NP-complete:

The sets _V_ 1 and _V_ 2 are non-empty and disjoint. In addition, their weight is bounded. Let's suppose that

For _k_ = 2, the ratio cut becomes:

[1.24]

As _V_ 1 and _V_ 2 are non-empty, _w_ ( _V_ 2) ≥ 1, then

and therefore

[1.25]

Thus, Π is NP-complete.

The problem of unconstrained graph partitioning optimization for the ratio cut is therefore NP-hard.

# 1.11. Conclusion

This chapter defines graph partitioning and describes different optimization problems associated with it. These can be grouped into two main categories, constrained graph partitioning problems and unconstrained graph partitioning problems.

The problem of constrained graph partitioning is the one that is most studied in the literature. It corresponds to the load distribution problem for parallel machines and is widely used in numerical analysis. The majority of graph partitioning methods have been created to solve this type of problems. However, most of these different methods can also be adapted to the unconstrained graph partitioning problems.

# 1.12. Bibliography

[ALP 95] ALPERT C.J., KAHNG A.B., "Recent directions in netlist partitioning: a survey", _Integration, the VLSI Journal_ , vol. 19, no. 12, p. 1-81, 1995.

[BAR 93] BARNARD S.T., SIMON H.D., "A fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems", _Proceedings of the 6th SIAM Conference on Parallel Processing for Scientific Computing_ , p. 711-718, 1993.

[BAR 94] BARNARD S.T., SIMON H.D., "A fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems", _Concurrency: Practice and Experience_ , vol. 6, p. 101-107, 1994.

[BEN 05] BENEZIT F., COUR T., SHI J., "Spectral segmentation with multiscale graph decomposition", _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '05)_ , vol. 2, p. 1124-1131, 2005.

[BIC 06] BICHOT C.E., "Metaheuristics versus spectral and multilevel methods applied on an air traffic control problem", _Proceedings of the 12th IFAC Symposium on Information Control Problems in Manufacturing (INCOM)_ , p. 493-498, May 2006.

[BIC 07] BICHOT C.E., "A new method, the fusion fission, for the relaxed k-way graph partitioning problem, and comparisons with some multilevel algorithms", _Journal of Mathematical Modeling and Algorithms (JMMA)_ , vol. 6, no. 3, p. 319-344, Springer, 2007.

[BIC 09] BICHOT C.E., "Co-clustering documents and words by minimizing the normalized cut objective function", _Journal of Mathematical Modeling and Algorithms (JMMA),_ vol. 9, no. 2, p. 131-147, Springer, 2009.

[CHA 07] Chardaire P., Barake M., McKeown G.P., "A PROBE based heuristic for graph partitioning", _IEEE Transaction on Computers_ , vol. 52, no. 12, p. 1707-1720, 2007.

[DHI 01] DHILLON I.S., "Co-clustering documents and words using bipartite spectral graph partitioning", _Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , p. 269-274, 2001.

[DHI 04] DHILLON I.S., GUAN Y., KULLIS B., A unified view of kernel k-means, spectral clustering and graph cuts, Report no. TR-04-25, University of Texas, Austin, 2004.

[DHI 07] DHILLON I.S., GUAN Y., KULIS B., "Weighted Graph Cuts without Eigenvectors: A Multilevel Approach", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 29, p. 1944-1957, 2007.

[DIN 01] DING C., HE X., ZHA H., GU M., SIMON H.D., "A min-max cut algorithm for graph partitioning and data clustering", _Proceedings of the 1st IEEE International Conference on Data Mining_ , p. 107-114, 2001.

[FEL 04] FELZENSZWALB P.F., HUTTENLOCHER D.P., "Efficient graph-based image segmentation", _International Journal of Computer Vision_ , vol. 59, no. 2, p. 167-181, 2004.

[GAR 79] GAREY M.R., JOHNSON D.S., _Computers and Intractability: A Guide to the Theory of NP-Completeness_ , W.H. Freeman and Company, 1979.

[GDA 01] GDALYAHU Y., WEINSHALL D., WERMAN M., "Self-organization in vision: stochastic clustering for image segmentation, perceptual grouping, and image database organization", _IEEE Transaction on Pattern Analysis and Machine Intelligence_ , vol. 23, no. 10, p. 1053- 1074, 2001.

[GIL 06] GIL C., BAÑOS R., MONTOYA M.G., GOMEZ J., "Performance of simulated annealing, tabu search, and evolutionary algorithms for multiobjective network partitioning", _Algorithmic Operations Research_ , vol. 1, p. 55-64, 2006.

[HAG 92] HAGEN L.W., KAHNG A.B., "A new approach to effective circuit clustering", _Proceedings of the IEEE/ACM International Conference on Computer Aided Design_ , p. 422- 427, 1992.

[HEN 98] HENDRICKSON B., "Graph partitioning and parallel solvers: Has the emperor no clothes? (Extended Abstract)", _Proceedings of the Workshop on Parallel Algorithms for Irregularly Structured Problems_ , p. 218-225, 1998.

[HOL 81] HOLYER I., "The NP-completness of some edge-partition problems", _SIAM Journal of Computing,_ vol. 10, no. 4, p. 713-717, 1981.

[HYA 73] HYAFIL L., RIVEST R., Graph partitioning and constructing optimal decision trees are polynomial complete problems, Report no. 33, IRIA-Laboria, Rocquencourt, France, October 1973.

[KAR 98] KARYPIS G., KUMAR V., "Multilevel k-way partitioning scheme for irregular graphs", _Journal of Parallel and Distributed Computing_ , vol. 48, no. 1, p. 96-129, 1998.

[KER 70] KERNIGHAN B.W., LIN S., "An efficient heuristic procedure for partitioning graphs", Bell System Technical Journal, vol. 49, no. 2, p. 291-307, 1970.

[MAR 04] MARTÍNEZ A.M., MITTRAPIYANURUK P., KAK A.C., "On combining graphpartitioning with non-parametric clustering for image segmentation", _Computer Vision and Image Understanding_ , vol. 95, no. 1, p. 72-85, 2004.

[MAR 05] MARTIN J.G., "Subproblem optimization by gene correlation with singular value decomposition", _Proceedings of the ACM Genetic and Evolutionary Computation Conference_ , p. 1507-1514, 2005.

[MAR 06] MARTIN J.G., "Spectral techniques for graph bisection in genetic algorithms", _Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation_ , p. 1249-1256, 2006.

[MEI 00] MEILA M., SHI J., "Learning segmentation by random walks", _Proceedings of the Neural Information Processing Systems_ , p. 873-879, 2000.

[PER 98] PERONA P., FREEMAN W., "A factorization approach to grouping", _Proceedings of the European Conference on Computer Vision, vol. 1406 of Lecture Notes in Computer Science_ , 1998.

[POT 90] POTHEN A., SIMON H.D., LIOU K.-P., "Partitioning sparse matrices with eigenvectors of graphs", _SIAM Journal on Matrix Analysis and Applications_ , vol. 11, no. 3, p. 430-452, 1990.

[RIE 94] RIESS B., DOLL K., Johannes F., "Partitioning very large circuits using analytical placement techniques", _Proceedings of the IEEE/ACM Design Automation Conference_ , p. 646-651, 1994.

[RON 05] RON D., WISHKO-STERN S., BRANDT A., An algebraic multigrid based algorithm for bisectioning general graphs, Report no. MCS05-01, Weizmann Institute of Science, 2005.

[RUM 02] RUMMLER A., APETREI A., "Graph partitioning revised - A multiobjective perspective", _Proceedings of the World Conference on Systemics, Cybernetics and Informatics_ , 2002.

[SCH 99] SCHLOEGEL K., KARYPIS G., KUMAR V., "A new algorithm for multiobjective graph partitioning", _Proceedings of the European Conference on Parallel Processing_ , p. 322-331, 1999.

[SEL 03] SELVAKKUMARAN N., KARYPIS G., "Multiobjective hypergraph-partitioning algorithms for cut and maximum subdomain-degree minimization", _Proceedings of the IEEE/ACM International Conference on Computer Aided Design_ , p. 726-733, 2003.

[SEL 06] SELVAKKUMARAN N., KARYPIS G., "Multiobjective hypergraph-partitioning algorithms for cut and maximum subdomain-degree minimization", _IEEE Transactions on Computer-Aided Design_ , vol. 25, no. 3, p. 504-517, 2006.

[SHI 98a] SHI J., BELONGIE S., LEUNG T., MALIK J., "Image and video segmentation: The normalized cut framework", _Proceedings of the IEEE International Conference on Image Processing_ , p. 943-947, 1998.

[SHI 98b] SHI J., MALIK J., "Motion segmentation and tracking using normalized cuts", _Proceedings of the 6th IEEE International Conference on Computer Vision (ICCV)_ , p. 1154- 1160, 1998.

[SHI 00] SHI J., MALIK J., "Normalized cuts and image segmentation", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 22, no. 8, p. 888-905, 2000.

[SIM 97] SIMON H.D., TENG S.-H., "How good is recursive bisection?", _SIAM Journal on Scientific Computing_ , vol. 18, no. 5, p. 1436-1445, 1997.

[WAN 01] WANG S., SISKIND J.M., "Image segmentation with minimum mean cut", _Proceedings of the 8th IEEE International Conference on Computer Vision_ , p. 517-524, 2001.

[WAN 03] WANG S., SISKIND J., "Image segmentation with ratio cut", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 25, no. 6, p. 675-690, 2003.

[WEI 89] WEI Y.C., CHENG C.K., "Towards efficient hierarchical designs by ratio cut partitioning", _Proceedings of the IEEE International Conference on Computer-Aided Design_ , p. 298-301, 1989.

[YAR 00] YARACK E., "An evaluation of move-based multiway partitioning algorithms", _Proceedings of the 19th IEEE International Conference on Computer Design: VLSI in Computers & Processors_, p. 363, 2000.

[ZHA 01] ZHA H., HE X., DING C.H.Q., GU M., SIMON H.D., "Bipartite graph partitioning and data clustering", _ACM Conference on Information and Knowledge Management_ , p. 25-32, 2001.

[ZHA 04] ZHAO Y., KARYPIS G., "Empirical and theoretical comparisons of selected criterion functions for document clustering", _Machine Learning_ , vol. 55, no. 3, p. 311-331, 2004.

* * *

1 Chapter written by Charles-Edmond BICHOT.

1 Energetic landscape is an expression used by extension from the analogy with physics, for the simulated annealing metaheuristics.
PART 1

Graph Partitioning for Numerical Analysis

# Chapter 2

# A Partitioning Requiring Rapidity and Quality: The Multilevel Method and Partitions Refinement Algorithms 1

This chapter presents the most popular method currently used to solve the constrained graph partitioning problem, the multilevel method. This method is based on several algorithms that can be used independently and/or in other contexts. They are local optimization methods, called refinement methods within a graph partitioning context, as well as direct partitioning methods such as region growing or the spectral method.

We will begin this chapter with an introduction to the multilevel method in section 2.1, then we will list the different levels of this method in section 2.2, before detailing each of these levels in three dedicated sections: sections 2.3, 2.4, and 2.5. We will conclude this chapter with the presentation of the spectral method in section 2.6.

# 2.1. Introduction

Initially, the multilevel method was created to accelerate existing graph partitioning tools [BAR 93]. However, it was soon recognized as a highly effective method that provided a more comprehensive vision of the graph [KAR 98a] than the classic partitioning methods used earlier, such as the region growing or partition refinement from Kernighan-Lin algorithms. However, these classic methods are still in use. Particularly, they are used by the multilevel method as local partitioning heuristics. Thus, the multilevel method was established as a "global" strategy using local graph partitioning algorithms. From this perspective, the multilevel method appears as a metaheuristic1 in graph partitioning, guiding different heuristics toward a global minimum, even if the iterative approach, so common to metaheuristics, is missing from this method, as provided below.

The multilevel method has been derived from _multigrid_ methods used in physics since the 1970s. This method has been used in parallel for the first time by S. Barnard and H. Simon [BAR 93] and by T. Bui and C. Jones [BUI 93]. However, thanks to the work of B. Hendrickson and R. Leland [HEN 95c] and of G. Karypis and V. Kumar [KAR 95], the multilevel method became popular in the graph partitioning field.

Because the multilevel method is very efficient to solve constrained graph partitioning problems, it has gradually replaced the use of spectral methods in the mid-1990s. In fact, spectral methods were at that time the most efficient methods for constrained graph partitioning, even though they were more difficult to implement than region growing or partition refinement.

The multilevel method is currently the most widely used method to solve the constrained graph partitioning problem; but its use tends to spread over unconstrained partitioning.

The literature about this method is abundant. We will therefore present the multilevel algorithms in this chapter, keeping the most popular ones, or those that we consider highly efficient, although less known.

# 2.2. Principles of the multilevel method

The multilevel method aims to answer a simple question: _how to rapidly create a partition in k parts of a large graph G, knowing that it is very expensive to deal with each vertex one by one?_

The natural answer to this question is to group vertices together in order to deal with groups of vertices, rather than independent vertices. This is the idea that originated the multilevel method.

The multilevel method is divided into three successive and well-distinct levels that act on the graph _G_ = ( _V , E_ ) ( _V_ is the set of vertices and _E_ the set of edges of the graph _G_ ):

– **Coarsening:** The coarsening phase is iterative. At each iteration, the vertices of the graph resulting from the previous iteration are grouped together to form a similar graph, but whose number of vertices is lower than the previous one. Thus, a family of graphs { _G_ 1, . . . , _G n_}, with _G_ 1 = G, is generated such that, for each graph _G i_+1, each of its vertices represents a group of vertices from the previous graph _G i_. The process ends when the graph is sufficiently small, or when the new graph generated has a size that is too close to the previous graph.

– **Partitioning:** This phase creates a partition   of the graph _G n_ in _k_ parts. To do so, the graph _G n_, resulting from the coarsening phase, is partitioned by using a partitioning heuristic (for example, a region growing method).

– **Uncoarsening and refinement:** The uncoarsening and refinement phase consists of projecting the division of the partition   onto the initial graph _G_. However, projecting directly   onto _G_ often creates a partition of mediocre quality, because this partition will be far from being locally optimal. Thus, the partition   of the graph _G n_ is first projected onto _G_ _n_ −1, then refined and becomes the partition  −1 of the graph _G_ _n_ −1. This process is repeated for all the graphs _G i_, _i_ = _n_ − 1 to 1, until we get the uncoarsening and refinement of the partition P1 k of the graph _G_ 1 = G. Thus, during uncoarsening and refinement, a family of partitions { , . . . ,   } is created, where each element   is the refined projection of the partition  +1 k of _G_ _i_ +1 onto the graph _G i_. Then, the final partition _P k_ =   , combines the properties of being good globally and locally.

The schema of the multilevel method is shown in Figure 2.1, where the three phases of the method are described. The coarsening phase of the multilevel method can give a global outline of the graph. Thus, even if the partitioning phase uses a local partitioning algorithm, the resulting partition will be good globally for the original graph, but certainly poor locally. The coarsening and refinement phase can improve the partition locally, while maintaining globally good structure of the partition.

Algorithm 2.1 presents the application of the multilevel method to graph partitioning. This algorithm remains very general and its effectiveness strongly depends on the methods used in the coarsening, partitioning, and uncoarsening and refinement phases. Many methods are used to achieve these three phases. In the following three sections, we will detail the most effective and the most common methods for each of these phases.

Since it is its usual domain of application, we will restrict ourselves, in the next three sections, to presenting the multilevel method in the constrained graph partitioning case, which consists of minimizing the cut of the partition and respecting the partitioning balance.

**Figure 2.1.** _The three phases of the multilevel algorithm_

* * *

**Algorithm 2.1.** The multilevel algorithm for graph partitioning

* * *

1: **procedure** MULTILEVEL(graph _G_ , number of parts _k_ )

2: _G_ 1 = _G_

3: _i_ = 1

4: **repeat**

5: _i_ ← _i_ \+ 1

6: _G i _= coarsening( _G_ _i_ −1)

7: **until** poor coarsening or _G_ _i_ +1 small enough

8:   = partition( _G i_, _k_ )

9: **for** _j_ = _i_ − 1 **to** 1 **do**

10: _P k_ = uncoarsening( +1, _G j_)

11:   = refinement( _P k_)

12: **end for**

13: **return**

14: **end procedure**

* * *

# 2.3. Graph coarsening

## 2.3.1. _Introduction_

The coarsening phase is a fundamental stage of the multilevel process. Indeed, if the quality of the grouping of vertices is not good, in other words, if it does not retain the global characteristics of the graph, then whatever the quality of the partitioning and refinement algorithms used, the quality of the final partition will be poor, because it will not be globally optimal.

As seen in the previous section, during the coarsening phase, a family of graphs { _G_ 1, . . . , _G n_}, with _G_ 1 = _G_ , is generated. This coarsening can be achieved in different ways, as provided below.

Generally, the coarsening phase consists of grouping together a set of vertices of the graph _G i_ in order to form a unique vertex in the graph _G_ _i_ +1. If we denote by _v_ as the resulting vertex in _G_ _i_ +1, then the subset of vertices of _G i_ that generates it is denoted by   . To ensure that the parts of the partitions balance between the different graphs _G j_, the weight of the vertex _v_ ∈ _G_ _i_ +1 will be equal to the sum of the weights of the vertices of   . The edges of the graph _G_ _i_ +1 will be formed in order to retain the connectivity information between the graphs, as well as to retain the cut between the parts of the partition corresponding to the different graphs _G j_. Thus, the weight of the edge ( _v_ 1, _v_ 2) will be equal to the sum of the weights of the edges linking   to  . This property is essential to retain the cut and the partitioning balance of a partition projected between the different graphs _G j_.

Since the aim of constrained graph partitioning is to minimize the cut between parts, the most intuitive approach to aggregate vertices, consists of grouping together the vertices connected by edges whose weight is maximal. Thus, the edges of low weight are more likely to be cut during the partitioning phase. Two possibilities exist regarding the size of the sets   formed: if each set   has at most two vertices, then we use the term _matching_ ; if the number of vertices of these sets can be greater than two, then we use the term _clustering_.

Coarsening by _clustering_ was used for relaxed circuits partitioning VLSI [HAG 92b]. It is particularly useful in the case of relaxed partitioning, since the weight of the internal edges of each part tends to be maximized, as it is the case for normalized cut as well as ratio cut. On the contrary, in the case of constrained graph partitioning, _matching_ is more often used.

The technique of coarsening by _matching_ consists of finding a maximal matching (the notion of matching is defined in the following section) of each graph _G j_ and grouping together the pairs of vertices of the matching [BUI 93, HEN 95c]. A maximum matching could contain more edges, however, it would take too much time for calculation when compared to the gain obtained [KAR 98a]. Because a maximal matching is used, the number of vertices of _G_ _i_ +1 cannot be lower than half the number of vertices of _G i_. Thus, at least _O_ ( _log_ (| _V_ |/ _n_ _′_ )) coarsenings will be necessary to obtain a graph of _n_ _′_ vertices. However, depending on the connectivity of the vertices of _G i_, the maximal matching can be much lower than | _V i_|/2. In that case, the ratio of the sizes of the sets of vertices between the two graphs, | _V i_|/| _V_ _i_ +1|, can be much lower than two. If the coarsening of a graph becomes too small, the coarsening process of the multilevel method can be halfed and we can jump to the partitioning of the last coarsened graph. However, the multilevel method remains efficient because the last coarsened graph, _G n_, has generally less vertices when compared with the initial graph _G_.

## 2.3.2. _Graph matching_

A graph matching is a set of edges of this graph that have no vertex in common.

DEFINITION 2.1. (MATCHING)– _Let G_ = ( _V , E_ ) _be a graph. The matching M of the graph G is a set of pairwise non-adjacent edges._

_The matching M is said to be maximal when each edge of the graph has one non-empty intersection with at least one edge of M._

_The matching is said to be maximum when it contains the highest possible number of edges._

_Note: Every maximum matching is also maximal_. The reciprocal is not necessarily true: for a linear graph with four vertices, the matching made of the edge formed by the two central vertices is _maximal_ , but not _maximum_. A graph can have several _maximum_ matchings (and _a fortiori_ several _maximal_ matchings).

The easiest algorithm to produce a maximal matching is a random iterative algorithm. At each iteration, an edge is randomly selected in the graph _G i_ = ( _V i_, _E i_) and is removed, along with all its adjacent edges. The vertex formed by the union of the two vertices of the chosen edge is added to _G_ _i_ +1, as well as the edges removed from _G i_. The complexity of such an algorithm is _O_ (| _E i_|).

Many parallel matching techniques are described in Chapter 4.

## 2.3.3. _Hendrickson-Leland coarsening algorithm_

Algorithm 2.2 presents the coarsening method [HEN 95c] for the software Chaco. Its principle is to randomly select an edge ( _v_ 1, _v_ 2) from the graph _G i_, aggregate its two vertices to form a vertex of the new graph _G_ _i_ +1, and then remove from _G i_ each edge adjacent to _v_ 1 and _v_ 2 to form the matching edges in _G_ _i_ +1. The result is a maximal matching of the graph. The complexity of this algorithm is in _O_ (| _E i_|).

* * *

**Algorithm 2.2.** Extended version of the Hendrickson-Leland graph coarsening

* * *

1: procedure Coarsening (Coarsened graph _G i_ = (Vi, Ei))

2: _E_ ← _E i_

3: _E i_+1 ← _E i_

4: _V i_+1 ← _V i_

5: **while** _E_ ≠ ∅ **do**

6: Randomly select an edge ( _v_ 1, _v_ 2) in _E_

7: Remove the edge ( _v_ 1, _v_ 2) from _E_ and _E_ _i_ +1

8: Remove _v_ 1 and _v_ 2 from _V_ _i_ +1

9: Add the new vertex _v_ = { _v_ 1, _v_ 2} to _V_ _I_ +1

10: _w_ ( _v_ ) = _w_ ( _v_ 1) + _w_ ( _v_ 2)

11: for all ( _v_ 1, _u_ ) ∈ _E_ **do**

12: if ( _v_ 2, _u_ ) ∈ _E_ **then**

13: _w_ ( _v_ 1, _u_ ) ← _w_ ( _v_ 1, _u_ ) + _w_ ( _v_ 2, _u_ )

14: Remove ( _v_ 2, _u_ ) from _E_ _i_ +1

15: **end if**

16: **end for**

17: **end while**

18: **return** _G_ _i_ +1 = ( _V_ _i_ +1, _E_ _i_ +1)

19: **end procedure**

* * *

## 2.3.4. _The Heavy Edge Matching (HEM) algorithm_

Another coarsening method is based on the search for a maximal matching but uses the weights of the edges of the graph [KAR 98a]. This method, named "Heavy Edge Matching" (HEM), is used in the partitioning tool METIS2. The aim is to find a maximal matching of the graph that minimizes the cut. For _G i_ = ( _V i_, _E i_), if we denote by _M i_ the matching of _G i_ that can generate _G_ _i_ +1, then the sum of the weights of the edges of _G_ _i_ +1 will be equal to the sum of the weights of the edges of _G i_, minus the sum of the weights of the edges of _M i_:

[2.1]

And yet, the lower the weight of the edges of the graph, the smaller the cut would be [KAR 95]. Thus, it is interesting to find a maximal matching where the edges have a maximum weight.

The HEM method consists of randomly selecting a vertex among the remaining vertices of the graph _G i_, that is to say the vertices that are not already in the matching _M i_ that we generate. Then the edge of maximal weight adjacent to this vertex is selected. This edge is added to _M i_ and its two vertices are joined to form a new vertex of _G_ _i_ +1. Then we must add the edges adjacent to these two vertices in _E_ _i_ +1. Algorithm 2.3 presents the corresponding pseudocode. The complexity of this algorithm is _O_ (| _E i_|).

* * *

**Algorithm 2.3.** The Heavy Edge Matching algorithm (HEM)

* * *

1: **procedure** Coarsening HEM (Coarsened graph _G i_ = ( _V i_, _E i_))

2: _V_ ← _V i_

3: _E_ _i_ +1 ← _E i_

4: _V_ _i_ +1 ← _V i_

5: **while** _V_ ≠ ∅ **do**

6: Randomly select a vertex _v_ 1 ∈ _V i_

7: Select the edge ( _v_ 1, _v_ 2) ∈ _E i_ of maximal weight

8: Remove _v_ 1 and _v_ 2 from _V_

9: Add the new vertex _v_ = { _v_ 1, _v_ 2} to _V_ _i_ +1

10: w(v) = w( _v_ 1) + w( _v_ 2)

11: **for each** edge ( _v_ 1, _u_ ) in _E_ **do**

12: **if** ( _v_ 2, _u_ ) ∈ _E_ **then**

13: _w_ ( _v_ 1, _u_ ) ← _w_ ( _v_ 1, u) + _w_ ( _v_ 2, _u_ )

14: Remove ( _v_ 2, _u_ ) from _E_ _i_ +1

15: **end if**

16: **end for**

17: **end while**

18: **return** _G_ _i_ +1 = ( _V_ _i_ +1, _E_ _i_ +1)

19: **end procedure**

* * *

The HEM method is used by other graph partitioning tools such as Scotch and Jostle. Although present in the METIS library, the HEM method is not used by default. A slightly different version, named _Sorted Heavy Edge Matching_ (SHEM) is used [KAR 98b]. The SHEM method sorts the vertices according to their degrees3 before applying the HEM method on this sorted list.

Currently, the SHEM method is probably the most efficient one for constrained partitioning. Indeed, sorting the vertices improves the quality of the final partition found by the multilevel method, at the cost of a computation time which is only slightly more important.

Once the coarsening phase is over through the HEM, SHEM, or Hendrickson- Leland algorithms, we have a coarsened graph _G n_ whose size is significantly smaller than the initial graph. The coarsened graph has often a few hundred vertices. The next stage of the multilevel method is to partition the graph. We shall see the way it happens in the following section.

# 2.4. Partitioning of the coarsened graph

## 2.4.1. _State-of-the-art partitioning methods_

The partitioning phase of the multilevel method aims to find a partition of the coarsened graph _G n_ into k parts. This partition will be used as the initial partition of the coarsening and refinement phase.

The partitioning phase of the multilevel method must follow one of the two existing partitioning processes, direct _k_ -partitioning or recursive bisection (these two partitioning processes are described in Chapter 1, page 18). Recursive bisection divides a graph using a bisection algorithm that will be repeated until k parts are obtained. Conversely, the direct _k_ -partitioning process only applies once the partitioning algorithm directly partitions the graph in _k_ parts. Later, we will see that the choice of a partitioning process at this stage will restrain the choice of the refinement method to be used during the uncoarsening and refinement phase (section 2.5, page 40).

The easiest solution to achieve the initial partition of the coarsened graph is to coarsen the graph until its number of vertices is equal to _k_. Thus, the coarsened graph is in itself the initial partition. This approach is used in several works: [GUP 97, WAL 00b, WAL 00a].

However, there are several disadvantages to this approach. The main disadvantage is that the aggregation stage produces a graph whose vertices are, for most of them, of very different weights [WAL 00a]. The uncoarsening and refinement phase will therefore have to be very effective to restore the partitioning balance of the initial partition. To counter this problem, M. Walshaw [WAL 00b] proposes a coarsening method which permits generation of vertices of similar sizes, whereas M. Gupta [GUP 97] suggests we successively uncoarsen, and then coarsen the vertices to restore the partitioning balance. Another drawback, often linked to the first one, appears when the graph _G_ is not connected. In that case, the coarsening phase can result in either a partition that is poorly distributed, or a number of vertices greater than the cardinality of the sought partition. However, despite the limitations that we have just stated, this approach has demonstrated its effectiveness [WAL 00a].

Most multilevel algorithms utilize a state-of-the-art graph partitioning algorithm to generate the initial partition. We will discuss the two methods which are commonly used:

– **the spectral method** can partition a graph, both in constrained and unconstrained ways, following a direct or recursive partitioning process. This method is detailed in section 2.6 (page 52);

– **region growing** is a fairly intuitive and effective partitioning method for small graphs. This method is usually used by following a recursive bisection-based partitioning process. Several variations of the region growing method are presented in the following section.

## 2.4.2. _Region growing methods_

### 2.4.2.1. _General description_

Region growing methods, or _graph growing_ , are greedy heuristics. They are very simple to implement and effective for small problems (several dozens to several hundreds of vertices). However, they are significantly less effective in bigger problems. Because of their qualities, these methods are used as underlying methods by multilevel methods. Inside the multilevel framework, these methods are used for creating partitions from the coarsened graphs.

These methods are used not only for a partitioning purpose, but also for the refinement or the load distribution of a partition [MEY 05, PEL 07], particularly in the case of diffusion algorithms [HU 98].

As partitioning methods, region growing methods follow the same blueprint. Starting from several vertices of the graph that form the premises of the parts of the partition, the algorithm consists of gathering the vertices of the graph in each of these sets by using a direct propagation method. In other words, a vertex can only be added to one of these sets if it is adjacent to at least one of the vertices of the set.

Region growing methods are very numerous, but as we have just explained, their blueprint is always the same. It would therefore be difficult to present all of them; that is why we will only discuss two of them: the _Graph Growing Partitioning_ algorithm and the _Greedy Graph Growing Partitioning_ algorithm. As we shall see, these two methods are graph bisection methods. To get a _k_ -partition ( _k_ > 2) from these algorithms, we can use the recursive bisection process presented in Chapter 1, page 18.

### 2.4.2.2. _The Graph Growing Partitioning algorithm (GGP)_

The _Graph Growing Partitioning_ algorithm (GGP) has been introduced in [KAR 98a]. This graph bisection method can be used as a partitioning method by a multilevel algorithm.

The GGP algorithm consists of iteratively generating a set _E_ that gathers half the vertices of the graph _G_ in terms of weight. This set _E_ is initialized by randomly selecting a vertex in the graph. During the execution of the algorithm, the vertices of the graph are divided into three sets: the set _E_ , the set of the vertices adjacent to _E_ , called border of _E_ and denoted by _F_ , and the set of the remaining vertices, denoted by _R_. At each iteration, the vertices adjacent to the vertices of the set _E_ are added to E, that is, _E_ = _E_ ∪ _F_. The process ends when _E_ contains a set of vertices which represents half of the total weight of the vertices of the graph. The algorithm then returns the bisection of the graph _G_ whose first part is _E_.

If this method is very easy to implement, the quality of the partition obtained depends mainly on the starting vertex chosen. As we have seen, in the case of a multilevel method, the number of vertices of the coarsened graph to partition is very low, often less than 200. Thus, this algorithm can be executed several times – around ten times for example – with a starting vertex which is always different. The bisection of the lower cut will therefore be chosen as a partition for the refinement phase of the multilevel method.

We have to underline that the partition gained by the GGP algorithm can be favorably refined by a refinement algorithm (refinement algorithms are described below in section 2.5, page 40) in order to reduce its cut.

### 2.4.2.3. _The Greedy Graph Growing Partitioning algorithm (GGGP)_

The GGP algorithm has been improved by its authors [KAR 98a] by adding the notion of cut gain of a vertex due to Kernighan-Lin (refinement method described below in section 2.5.2, page 41). This improvement has resulted in the _Greedy Graph Growing Partitioning_ algorithm (GGGP).

Algorithm 2.4 presents the GGGP algorithm. This algorithm has the same iterative structure as the GGP algorithm and also distributes the vertices of the graph in the three sets _E_ , _F_ , and _R_ described previously. It also includes differently the vertices in _E_. As in the Kernighan-Lin algorithm, to each vertex _v_ of _F_ we associate a gain relating to the reduction of the cut which would result from the insertion of _v_ in _E_. Thus, the vertices of the border of the partition can be sorted according to their gain.

As in the case of the GGP algorithm, the GGGP algorithm starts with the initialization of the set _E_. _E_ is initialized as a set that only contains a vertex randomly selected in _V_. The sets _R_ and _F_ can then be initialized. Each iteration of the algorithm consists of inserting in _E_ the vertex of F that can reduce the most cut. This vertex, denoted by _v_ , is the vertex of maximal gain of _F_. Then each vertex of _R_ adjacent to _v_ is moved to _F_ and its gain is calculated. Similarly, each vertex of F adjacent to v has its gain recalculated and the next iteration can therefore start. The algorithm ends when the total weight of _E_ is equal to half the weight of _V_.

As noted by G. Karypis and V. Kumar, in order to make the algorithm work faster, the data structure of Fiduccia-Mattheyses (section 2.5.3, page 46) can be used. The only difference is that instead of calculating the gain for each unblocked vertex, the gain is calculated for the vertices of _F_ only.

The GGGP algorithm is less subject to the choice of the initial vertex of _E_ than the GGP algorithm. However, in the case of a multilevel method and as for GGP, this

* * *

**Algorithm 2.4.** The Greedy Graph Growing Partitioning algorithm (GGGP)

* * *

1: **procedure** GGGP (graph _G_ = ( _V, E_ ))

2: Randomly select a vertex v0 ∈ _V_

3: _E_ ← { _v_ 0}

4: _R_ = _V_ − { _v_ 0}

5: _F_ ← { _v_ ∈ _R_ such that ( _v_ , _v_ 0) ∈ _E_ }

6: Compute the gains of _F_

7: **while** _w_ ( _E_ ) < 1/2 _w_ ( _V_ ) **do**

8: Select the vertex _v i_ ∈ _F_ of maximal gain

9: Move _v i_ from _F_ to _E_

10: **for each** edge ( _v_ , _v i_) in _E_ **do**

11: **if** _v_ ∈ _F_ **then**

12: update the gain of _v_

13: **else if** _v_ ∉ _E_ **then**

14: add _v_ to _F_

15: compute the gain of _v_

16: **end if**

17: **end for**

18: **end while**

19: **return** _E_

20: **end procedure**

* * *

algorithm can be started several times in order to reduce the importance of the choice of the initial vertex. For example, GGGP is started four times in [KAR 98a]. The authors consider that, even in that case, GGGP remains as fast as GGP, while producing better quality bisections.

# 2.5. Uncoarsening and partitions refinement

## 2.5.1. _Presentation of the uncoarsening and refinement phase_

The uncoarsening and refinement phase of the multilevel method consists of progressively projecting onto the initial graph the partition of the coarsened graph generated during the partitioning phase. This stage would be very simple if a direct projection of the coarsened graph partition onto the initial graph, led to a locally optimal partition. However, as G. Karypis and V. Kumar notice [KAR 95], a locally good partition of a coarsened graph _G_ _i_ +1 from _G i_ is often of lower quality for the graph _G i_. Different refinement algorithms are therefore used to locally improve the cut of the partition after each projection.

The first phase of the multilevel method, the coarsening phase, has constructed a family of coarsened graphs { _G_ 1, . . . , _G n_}. The uncoarsening and refinement phase consists of projecting recursively the partition of a coarsened graph _G_ _i_ +1 onto its parent _G i_ and then applying a refinement algorithm to this last partition. Thus, a family of partitions { , . . . ,  } is generated, with each element   being the partition of  +1 k projected onto _G i_ followed by a refinement. The iteration ends when the partition   projected onto the initial graph _G_ 1 is refined. The resulting partition   therefore associates a globally good configuration, due to the multilevel method, with a locally efficient division due to the refinement algorithm.

Numerous refinement methods exist. However, all the refinement algorithms have two characteristics in common: they have to start from an existing partition and the optimization has to be local. Many of these methods are based on the Kernighan-Lin algorithm KER 70]. However, other methods exist, such as the "Helpful-Sets" [DIE 95, DIE 96, SCH 03, MON 04]. Simulated annealing and other meta-heuristics have also been used as refinement methods [MAR 96, DIE 96, BAÑ 03] (for more details on these methods, refer to[ Chapter 6). Recently, a new refinement method, based on diffusion, has emerged PEL 07] (this method is presented in [Chapter 4, section 4.6.4.2, page 104).

As specified by B. Hendrickson and R. Leland [HEN 95b], the refinement algorithm used in multilevel methods has to be fast and effective. The Kernighan-Lin algorithm, presented in the following section (section 2.5.2), adopt both criteria. Besides, the Kernighan-Lin algorithm is more effective if the initial partition to refine is of good quality, which is the case for multilevel methods.

To improve the speed of the Kernighan-Lin algorithm, the implementation described by C. Fiduccia and R. Mattheyses [FID 82] is systematically used. This implementation is described in section 2.5.3.

These two algorithms have been developed to refine graph bisections. But as we have seen, the multilevel method can follow the recursive or direct partitioning process. In the first case, the refinement algorithms that we have just mentioned are perfectly suitable and are applied after each bisection has been found. However, in the case of a direct _k_ -partitioning process, these refinement algorithms have to be adopted, as we shall see in section 2.5.4.

Finally, we shall describe two refinement algorithms that currently seem to be most effective, the GKLR algorithm by G. Karypis and V. Kumar [KAR 98d] (in section 2.5.5) and the algorithm by C. Walshaw and M. Cross [WAL 00a] (in section 2.5.6).

## 2.5.2. _The Kernighan-Lin algorithm_

### 2.5.2.1. _General principle_

The Kernighan-Lin algorithm [KER 70] can refine the bisection of a graph previously obtained. It is therefore a local optimization algorithm. The main idea of this algorithm is to find two subsets of vertices of equal size, each in a part of the bisection, such that their exchange reduces the cut of the bisection. In order to find "a locally optimal partition" [KER 70], the algorithm proceeds as follows: starting from an existing bisection, the algorithm successively exchanges two subsets of the bisection, until no more subset reducing the cut can be found. The final bisection is therefore the minimal cut bisection found during the iteration process of the algorithm.

### 2.5.2.2. _The algorithm_

Initially, let us suppose that we are looking for a perfectly balanced bisection (i.e. of unit partitioning balance) and that all the vertices have the same weight. We will see in the following sections the way to adapt the algorithm to parts of different sizes.

To generate the two subsets of the bisection to exchange, the authors have introduced two notions: the external cost and the internal cost. The internal cost _I_ ( _v i_) of a vertex _V i_ of the part _V i_ of the bisection is defined as the sum of the weights of the edges adjacent to _V i_ whose second vertex is in _V i_:

[2.2]

The external cost _E_ ( _v i_) of a vertex _V i_ of _V i_ is defined as the sum of the weights of the edges between _V i_ and the vertices that do not belong to _V i_:

[2.3]

Let _D_ ( _v i_) be the difference between the external cost and the internal cost of the vertex _v i_:

[2.4]

Consider two vertices each belonging to a different part of the bisection _P_ = ( _v_ 1, _v_ 2), _v_ 1 ∈ _v_ 1 and _v_ 2 ∈ _v_ 2. The gain, denoted by _g_ , resulting from the exchange of _v_ 1 and _v_ 2 is:

[2.5]

For each vertex _V i_ ∈ _V i_, the algorithm calculates the difference between the external cost and the internal cost of _v i_, that is _D_ ( _v i_). The values D( _v i_) are distributed in two sets _D_ 1 and _D_ 2, according to the part which _V i_ belongs to.

The Kernighan-Lin algorithm, presented by Algorithm 2.5, is developed around an iterative loop, called by its authors _phase 1 optimization_ or "pass", that allows us to look for the two subsets of the bisection to exchange. At each pass, the construction of these two subsets is done by successive iterations, by adding an element to each

* * *

**Algorithm 2.5.** The Kernighan-Lin algorithm

* * *

* * *

subset at each iteration. Consider the algorithm at the q iteration. The algorithm starts by looking for two vertices each belonging to one part of the bisection _P_ = ( _v_ 1, _v_ 2), that maximizes _g_ (see equation [2.5]). To do so, each set _D_ 1 and _D_ 2 is sorted:

[2.6]

Once the sets are sorted, only a small number of pairs of vertices will be considered. Indeed, if two pairs ( _v_ 1, _i_ ) and ( _v_ 2, _j_ ) are found such that _D_ ( _v_ 1, _i_ ) + _D_ ( _v_ 2, _j_ ) is less than or equal to the maximum gain previously found, then there cannot be another pair of vertices (( _v_ 1, _k_ ), ( _v_ 2, _l_ )) with _k_ ≥ _i_ and _l_ ≥ _j_ , with a gain greater than this maximum gain. Thus, two vertices   ∈ _D_ 1 and   ∈ _D_ 2 of gain _g q_ are selected. They are then locked, which means that it will not be possible to select them again during the rest of this pass. The values _D_ ( _v i_) of the sets _D_ 1 and _D_ 2 are recalculated for the unlocked vertices according to the new bisection ( _V_ 1 − { }, _V_ 2 − { }). Let   and   be the new sets:

[2.7]

The procedure is then iterated by starting from the bisection ( _V_ 1 − { }, _V_ 2 − { }) and the sets   and  . The iteration ends when there are no more vertices to exchange. As the parts of the bisection are of equal size, each part is then empty.

The gain resulting from a pass, that is to say from the exchange of the set of vertices {  , . . . ,  } and {  , . . . ,  }, is:

[2.8]

After the last iteration, the total gain of the exchange is: _G_ (| _V_ |/2) = 0.

At the end of the pass, we look for the value   that maximizes the gain _G_ ( _q_ ): _G_ ( ) = max _q_ _G_ ( _q_ ). If the maximum gain _G_ ( ) is strictly positive, then the two corresponding sets of vertices,   and  , are exchanged.

The Kernighan-Lin algorithm proceeds by succesive passes as long as they allow us to find two sets of vertices to exchange. The final bisection has a cut equal to the initial cut minus the sum of the gains obtained by the different passes.

### 2.5.2.3. _Improvements suggested by B. Kernighan and S. Lin_

To find a graph bisection with a cut even lower, the authors have recommended to repeat the algorithm by initializing it with different bisections. They also propose a different approach, which consists in disrupting the solution previously obtained by the algorithm, by randomly exchanging a few vertices of each part. This algorithm is then applied to this new bisection.

The Kernighan-Lin algorithm is faster and more efficient if its initial partition is good. Indeed, if the initial partition is good, then the number of iterations of the algorithm will be low, because it will be impossible to find subsets to exchange. Moreover, the optimization being local, a very significative reduction of the cut is unlikely. Thus, to get a minimal cut, it is best to start from a bisection of low cut. The choice of the method generating the initial bisection is therefore very important. The authors do not recommend a specific method to create the initial bisection. However, it can be easily produced by region growing algorithms (section 2.4.2). This is the solution used in some multilevel methods, including [KAR 98a].

### 2.5.2.4. _Complexity_

The complexity of the Kernighan-Lin algorithm is its main drawback. The calculation of the external and internal costs has a complexity of _O_ (| _E_ |2). The time taken to find the two vertices to exchange during a pass is in _O_ (| _E_ |2 _log_ | _E_ |). The complexity of the Kernighan-Lin algorithm is therefore in _O_ (| _E_ |2 _log_ | _E_ |). In the following sections, we will see other algorithms which can reduce this complexity.

However, to accelerate the selection of the two vertices to exchange, B. Kernighan and S. Lin suggest that the lists _D_ 1 and _D_ 2 should not be sorted, and instead look for the greatest value of each of these two lists and exchange the vertices corresponding to these values. Since this search is linear (i.e. in _O_ (| _E_ |)), the complexity of the Kernighan-Lin algorithm will be in _O_ (| _V_ |2). The downside of this new search is that it does not allow us to find the two vertices of maximal gain. Indeed, this search does not take into account the weight between the two vertices present in the equation [2.5].

### 2.5.2.5. _Parts of different sizes_

Let's consider the case of a bisection in a part of _n_ 1 elements and in another part of _n_ 2 elements, where _n_ 1 < _n_ 2. The problem is to refine the bisection to obtain one that will have parts of at least _n_ 1 vertices and at the most _n_ 2 vertices. B. Kernighan and S. Lin propose to adapt their algorithm to such a problem. The number of pairs of vertices that will be exchanged at each pass is reduced to _n_ 1. To the part of size _n_ 1, _n_ 2 − _n_ 1 "virtual" vertices are added. These virtual vertices have a weight equal to the other vertices, but are not connected by any arc to other vertices. The cost, as the gain of the transfer of one of these vertices from a part to another, is therefore zero. After the Kernighan-Lin algorithm has been applied, the virtual vertices are removed from the refined bisection. The bisection obtained has 2 parts of at least _n_ 1 vertices and at the most _n_ 2 vertices.

### 2.5.2.6. _Vertices of different weights_

Until now, only vertices of equal weight have been considered. To overcome this restriction, the authors suggest that each vertex of weight _p_ > 1 be replaced by _p_ vertices of unit weight connected together. The problem of this approach is that it strongly increase the size of the problem instances and that some vertices may be divided between the 2 parts. This solution is therefore not viable for most of the problems.

### 2.5.2.7. _Other improvements of the Kernighan-Lin algorithm_

Many improvements of the Kernighan-Lin algorithm have been suggested. Because this algorithm has been developed for the bisection of VLSI circuits and because they are often modeled with hypergraphs, many improvements affect hypergraphs [FID 82, KRI 84, SAN 89, HAG 97, ZHA 02, PAP 07]. However, we owe to one of these chapters, [FID 82], the systematically used implementation to reduce the complexity of this algorithm, both in graph and hypergraph partitioning cases. We will study this implementation in the following section.

In the case of graph partitioning, and regardless of the chapters on multilevel methods, only a few chapters have introduced improvements of the Kernighan-Lin algorithms [BUI 89, DUT 93, PEL 96, RAN 01]. However, chapters introducing multilevel methods often suggest a new Kernighan-Lin refinement algorithm in the case of recursive bisection [KAR 98a] or direct _k_ -partitioning [HEN 95c, WAL 97, KAR 98d, KAR 98c, WAL 00a].

**Figure 2.2.** _Implementation of the Fiduccia-Mattheyses gains table. Table of vertices points to double-linked lists connected to the gain of the corresponding vertex_

## 2.5.3. _Fiduccia-Mattheyses implementation_

C. Fiduccia and R. Mattheyses [FID 82] propose a refinement algorithm based on the Kernighan-Lin algorithm, but for hypergraph partitioning. Still concerned with a bisection problem, they reuse the concept of gain introduced [KER 70] and presented in section 2.5.2. The main focus of this chapter is the description of the data structure used to update the gains of the vertices.

The gain of a vertex represents the reduction of the cut generated by the transfer of this vertex to the other part of the bisection. Formally, the gain _g_ ( _v_ ) of a vertex _v_ is equal to the difference between the external cost and the internal cost of the vertex. According to equation [2.4], _g_ ( _v_ ) = _D_ ( _v_ ).

The data structure that can maintain the gain of the vertices updated is shown in Figure 2.2. This data structure consists of maintaining, for each part of the bisection, a sorted table of the gains of the vertices. A doubly-linked list is associated with each cell of the gains table. This doubly-linked list contains the vertices whose gain corresponds to the rank of the list in this gains table. Finally, each cell of the vertices table points at its vertex, and belongs to one of the doubly-linked lists. Thus, accessing a vertex runs in constant time, and its transfer from one gain to another during the gains update also runs in constant time.

* * *

**Algorithm 2.6.** Fiduccia-Mattheyses algorithm adapted to graph partitioning

* * *

* * *

**Algorithm 2.6** presents the adaptation of the Fiduccia-Mattheyses algorithm to graph partitioning.

The gain initialization for each vertex has a complexity of _O_ (| _E_ |). Similarly, because the vertices gain update is constant, each pass of the Kernighan-Lin algorithm is _O_ (| _E_ |). Thus, the general complexity of the algorithm is linear as compared to the size of E, which is a great improvement when compared to the Kernighan-Lin algorithm, which was _O_ (| _E_ |2 _log_ | _E_ |).

## 2.5.4. _Adaptation to direct k-partitioning_

The Kernighan-Lin algorithm that uses the Fiduccia-Mattheyses implementation is efficient and fast, but it is meant to refine a bisection. However, in many cases, the partition to refine has a number of parts _k_ strictly greater than two and not always even. It is therefore useful to have an efficient version of this algorithm for direct _k_ -partitioning.

One of the first adaptations of the Kernighan-Lin algorithm to direct _k_ -partitioning is due to L. Sanchis [SAN 89, SAN 93]. However, this adaptation was made in a hypergraph partitioning context.

An adaptation of this algorithm to _k_ -partitioning is introduced by B. Hendrickson and R. Leland [HEN 95c]. However, this adaptation is expensive in terms of space and time. Indeed, it uses _k_ ( _k_ − 1) gains tables of the Fiduccia-Mattheyses implementation (section 2.5.3) and has a complexity _O_ ( _k_ | _E_ |), as compared to the complexity of the Fiduccia-Mattheyses algorithm, which is _O_ (| _E_ |). Thus, this adaptation can be used only when _k_ is low.

To overcome these drawbacks, G. Karypis and V. Kumar [KAR 98d] propose another approach. It still uses the gain concept and the Fiducci-Mattheyses implementation, but has a complexity that is independent of the number of parts of the sought partition. In their algorithm, only one gains table is used. However, according to the authors, though their refinement method is faster, it is less efficient than the one from B. Hendrickson and R. Leland. Section 2.5.5 describes this method in more detail.

## 2.5.5. _Global Kernighan-Lin Refinement_

The refinement algorithm _Global Kernighan-Lin Refinement_ (GKLR) [KAR 98d] uses the same data structure as the one from the Fiduccia-Mattheyses implementation, which can maintain the vertices gain updated (section 2.5.2.2), the only difference being that only one gains table is used. As all the Kernighan-Lin algorithms, this algorithm is iterative and each iteration is called a "pass" (section 2.5.2.2). Each pass consists of moving a set of vertices to other parts in order to reduce the cut of the partition.

Algorithm 2.7 describes the refinement process of GKLR.At the beginning of a pass, the gain of each vertex is calculated according to equation [2.4], but by considering only 2 parts at once: the part of the vertex and one part adjacent to this vertex. In this algorithm, the gain of a vertex is the maximum gain of all the gains calculated between this vertex and the parts adjacent to this vertex. If the gain is positive, it is retained, otherwise the vertex is locked. The authors have chosen to retain only the vertices whose gain is positive at the beginning of a pass, because it has the advantage of significantly reducing the time of a pass. However, it has the disadvantage of not being able to reach the local minima.

Since the data structure of Fiduccia-Mattheyses is formed of exchangeable vertices and their corresponding gains, an iteration on the unlocked vertices begins. Each of these iterations start with the selection of the first vertex of the doubly-linked list of higher gain in the gains table. Let _v_ be this vertex; the algorithm will select the part adjacent to the part of _v_ that maximizes the gain of _v_ , while maintaining the partitioning balance. Thus, the vertex _v_ can be moved to a part of non-maximal gain, which does not correspond to the gain of the gains table. Once the vertex _v_ is moved to its new part, it is locked and the gains of the unlocked adjacent vertices are updated. At the end of the iteration, the vertex _v_ is locked. The iteration on the unlocked vertices carries

* * *

**Algorithm 2.7.** The Global Kernighan-Lin Refinement algorithm

* * *

1: **procedur** e GKLR (graph _G_ = ( _V, E_ ), partition _P k_ , max balance maxbal)

2: **repeat** "pass"

3: Create the gains table and the set of unlocked vertices _V v_

4: **while** _V v_ ≠ ∅ **or** _dep_ gains have not been negative **do**

5: Select the vertex _v_ of greatest gain. Let _V j_ ∈ _P k_ such that _v_ ∈ _V j_

6: **for all** _V i_ ∈ _P k_ − { _V j_} such that _V i_ ∪ { _v_ } maintains maxbal **do**

7: Compute the gain of adding _v_ to _V i_

8: **end for**

9: Select the part _V i_ ∈ _P k_ of maximal gain g for the vertex _v_

10: Record the transfer of v to _V i_ and the corresponding gain _g_

11: **for all** unlocked vertex _v_ _′_ adjacent to _v_ **do**

12: Update the gains table for _v_ _′_

13: **end for**

14: Lock _v_

15: **end while**

16: Select the set of vertices to transfer that maximizes the gain _g_

17: **if** _g_ > 0 **then**

18: Alter _P k_ by adding the set of vertices to transfer

19: **end if**

20: **until** _g_ ≤ 0 or a predefined number of passes is reached

21: **return** _P k_

22: **end procedure**

* * *

on until there are unlocked vertices or until a predefined number, _dep_ , of succesive moves of vertices has not led to at least improvement of the cut. During the iteration, the gains are updated and can become strictly negative. Thus, the gain of the partition can be reduced.

Once this iteration is complete, the set of vertices, if any, that has allowed to reduce most of the cut, is moved to the corresponding parts. The move of these vertices ends the pass.

The iteration on the passes ends when a predefined number of passes have been executed or when a pass has not reduced the cut of the partition.

One of the main drawbacks of this refinement algorithm is that it is difficult to correctly refine partitions that do not already maintain the partitioning balance. This algorithm is often used in multilevel methods. However, the uncoarsening and refinement phase of the multilevel methods does not always produce a partition maintaining the partitioning balance.

## 2.5.6. _The Walshaw-Cross refinement algorithm_

The Walshaw-Cross refinement algorithm [WAL 00a] has been inspired by other methods. The algorithm is based on the gain idea from Kernighan-Lin, and uses the Fiduccia-Mattheyses implementation to keep the gains updated. The uncoarsening and refinement phase of this algorithm is close to the GKLR algorithm [KAR 98d] (section 2.5.5). Like this algorithm, the vertices are moved one by one and only one gains table is necessary. However, the authors have noticed that the GKLR algorithm could not restore the balance of the partitions which were poorly balanced, so they have used a load distribution algorithm to balance the parts between themselves. This algorithm is based on a distribution method described in [HU 98]. This algorithm does not place constraint on partitioning, but aims to perfectly restore the balance for the partition. It can specify which weight to transfer between the parts of the graph, whereas the GKLR algorithm used later specifies which vertices to transfer.

Distribution methods minimize the Euclidian norm of the weight transferred bewteen the parts. The underlying idea of these methods is to find which weight to transfer between adjacent parts to balance the partition. The weight transfer only affects adjacent parts, in order to avoid the increase of the cut of the partition. To do so, the distribution algorithm described in [HU 98] attempts, for a partition _P k_ = { _V_ 1, . . . , _V k_}, to solve the linear system:

[2.9]

where _b_ is a vector of size _k_ = | _P k_| such that each of these components is:

[2.10]

and where _M Lap_ is the Laplacian matrix of the partition _P k_ used as a new graph, with:

[2.11]

The weight to transfer between the parts _V i_ and _V j_ is then given by the difference _x i_ − _x j_. It is worth noting that this weight to transfer is directed, that is to say that the weight to transfer between _V j_ and _V i_ is the opposite of the weight to transfer between _V i_ and _V j_ . To solve this linear system, the authors use the iterative algorithm of the conjugate gradient. This algorithm is easy to initialize, as we know the first eigenvector of _M Lap_ (section 2.6.2), which is unitary.

The authors of [HU 98] suppose that the graph is connected, otherwise they suggest using the algorithm introduced in [SON 94] to re-create these flows of weight to transfer.

Once the distribution algorithm has found the flows of weight to transfer, a GKLR algorithm is used, but with several variants:

– at the beginning, only the vertices on the border of the partition are ordered in the gains table. Then, during the iteration, when a vertex is found on the new border, it is added to the gains table. Conversely, a vertex, which is, no longer on the border is removed from the gains table;

– the gains table used is sorted by gain and then by weight, and each combination gain/weight is connected to a doubly-linked list. If the gain is positive, then the vertices are arranged in ascending order of weight, otherwise, they are arranged in descending order of weight. Indeed, it is best to transfer three vertices of weight 1 and gain 1 than a vertex of weight 3 and gain 1. Conversely, it is best to move a vertex of weight 3 and gain 1 than three vertices of weight 1 and gain 1;

– moving a vertex does not take the gain into account. If the weight of the part that was, added to the weight of the vertex maintains the partitioning balance, then the transfer is accepted. Even, if the transfer of the vertex reduces the flow of weight to transfer, it is still accepted;

– at the end of each iteration, and not at the end of each pass, the new refined partition _P r_ is compared with the best partition _P_ previously obtained and is retained instead of the latter if it meets one of the following conditions: _P_ does not maintain the partitioning balance and _bal_ ( _P r_) < _bal_ ( _P_ ); or _cut_ ( _P r_) < _cut_ ( _P_ ); or _cut_ ( _P r_) = _cut_ ( _P_ ) and _bal_ ( _P r_) < _bal_ ( _P_ );

– the vertices adjacent to several parts can be selected several times. Let's consider a vertex adjacent to _V i_ and _V j_ such that its gains are respectively _g i_ > _g j_ . The vertex is initially placed in the gains table at rank _g i_. If the transfer of this vertex to _V i_ is not accepted, then the vertex is moved to rank _g j_ , and when this vertex is selected again, it is tested for a transfer to _V i_ then _V j_ . If none of these transfers is accepted, then the vertex is locked and removed from the gains table.

The idea to use only the vertices on the borders at first, instead of all the vertices, has been introduced in [HEN 95c], and more precisely in [WAL 95]. This idea can extend to the case of vertices at some distance from the border, with distance being the concept of shortest path between two vertices. Another extension would be to limit the course of the Kernighan-Lin algorithm to these vertices only. Indeed, in the case of multilevel algorithms, the partition to refine is globally good, as recollected in [KAR 95]. The refined partition is therefore likely to be very close to the original partition. Thus, it would reduce the execution time of the algorithm. Moreover, this technique would maintain the "globally good" structure of the partition (in an energetic well), during the uncoarsening and refinement phase of the multilevel algorithm. Indeed, a strong refinement of the partition, while it has not yet been projected onto the orginal graph, might remove the partition from the energetic well, toward a locally better solution at this projection stage, but globally less good for its projection on the orginal graph.

# 2.6. The spectral method

## 2.6.1. _Presentation_

Using the spectral method to solve graph partitioning problems is very ancient. The first few chapters offering such an approach are due to W. Donath and A. Hoffman [DON 72, DON 73]. The spectral method was widely used to solve graph partitioning problems before multilevel methods were developed. Its importance for graph partitioning was such that one of the first chapters offering the use of a multilevel method only uses the latter to simplify the computation required by the spectral method [BAR 93, BAR 94]. Subsequently, the balance of power was reversed, and the spectral method emerged as one of the tools used in the multilevel methods [HEN 95b].

Many chapters introduce the use of the spectral method to graph partitioning, for both constrained [HEN 95b, BAR 94, HEN 93, POT 90] and unconstrained graph partitioning problems [HAG 92a, DIN 01, DHI 04a, DHI 04b].

The theoritical results linked to the spectral method applied to graph partitioning are also numerous. They can set a limit lower than the cut of a partition [DON 73, BEZ 99, ELS 03], or estimate the quality of a graph separator4 [GUA 98]. N. Alon and V. Milman [ALO 85] prove that obtaining a cut from the spectral method has, at worst, a cost which is a quadratic of the cost of the optimal partition.

The spectral method has been named after the spectral theorem of linear algebra. This theorem can maintain the diagonalization of real symmetric matrices. It also justifies the decomposition of real symmetric matrices in eigenvalues within an orthonormal basis of eigenvectors. And yet, the graph partitioning problem can, at the cost of two approximations (section 2.6.5, page 56), be reduced to the resolution of a numerical system _Mx_ = _λx_. Solving this numerical system consists of finding an orthogonal basis of eigenvectors of the matrix _M_.

## 2.6.2. _Some results of numerical system_

In this section, we shall see how the graph partitioning problem can be solved by finding a family of eigenvalues and a family of eigenvectors of a matrix. The results presented in this section have been the subject of many publications, mainly in the case of unweighted graphs [POT 90, HAG 92a, HEN 93]. However, we will generalize these concepts to extend them to the case of weighted graphs [MOH 91, MOH 97].

Before proceeding, we shall introduce some definitions in numerical analysis, which will be used in this section.

DEFINITION 2.2. (EIGENVECTOR, EIGENVALUE)– _Let E be a vector space on a division ring K . Let u be an endomorphism of E. Assume a vector x of E and λ in K such that u(x) = λx is called eigenvector of u. In that case, λ is called eigenvalue of u._

If we consider an endomorphism on real numbers, and _M_ a matrix of this endomorphism, then we call eigenvector of _M_ every vector _x_ (i.e. column matrix) of real numbers that verifies ∃ _λ_ ∈ _R, Mx_ = _λx_.

DEFINITION 2.3. (POSITIVE SEMI-DEFINITE MATRIX)– _Let M be a real symmetric matrix of order n. It is said to be positive semi-definite if it verifies one of the two following equivalent properties:_

_1) for every non-zero column matrix x of n real elements, we have x T Mx_ ≥ 0;

_2) all the eigenvalues of M are positive or null._

We will now consider the Laplacian matrix of a graph. We have to define the adjacency matrix and the degree matrix of a graph.

DEFINITION 2.4. (ADJACENCY MATRIX OF A GRAPH)– _Let G_ = ( _V, E_ ) _be a simple graph. The_ | _V_ | × | _V_ | _matrix M Adj_ , _such that_ ∀( _i, j_ ) ∈ {1, . . . , | _V_ |}2:

[2.12]

_is called adjacency matrix of the graph G._

DEFINITION 2.5. (DEGREE MATRIX OF A GRAPH)– _The_ | _V_ | × | _V_ | _matrix M Deg_ , _such that_ ∀( _i, j_ ) ∈ {1, . . . , | _V_ |}2:

[2.13]

_is called degree matrix of the graph G._

DEFINITION 2.6. (LAPLACIAN MATRIX OF A GRAPH)– _Let G_ = ( _V, E_ ) _be a simple graph. The matrix_

[2.14]

_is called Laplacian matrix of G_ [MOH 97].

We have the following property:

PROPERTY 2.1.– _The Laplacian matrix M Lap of an undirected and positively weighted graph G is positive semi-definite._

_Proof_ : Let _x_ be a vector of size _n_. We have for each _i_ ∈ {1, . . . , _n_ }:

[2.15]

Then,

[2.16]

And because the graph is undirected, i.e. ∀( _i, j_ ) ∈ 1, . . . , _n_ 2, _w_ ( _i, j_ ) = _w_ ( _j, i_ ), we have:

[2.17]

Because the graph is positively weighted, _M Lap_ is positive semi-definite.

Consider that we attempt to find a bisection of G in two subsets of vertices _v_ 1 and _v_ 2. We shall see in the following section (section 2.6.3) how to generalize this method to 2i-partitioning.

Let _G_ = ( _V, E_ ) be a graph and _x_ a vector of size _n_ such that:

[2.18]

If we consider the cut between _V_ 1 and _V_ 2, as defined in Chapter 1, page 13:

[2.19]

And if we use the result of equation [2.17], then:

[2.20]

Thus, minimizing the cut of a bisection aims at finding a vector _x_ that minimizes _x T MLapx_.

In other words, this problem consists in solving the linear system:

[2.21]

by finding the smallest real number _λ_. As we have seen at the beginning of this section, it aims at finding the minimal eigenvalue of the Laplacian matrix _M Lap_ of the graph _G_ , as well as its corresponding eigenvector.

## 2.6.3. _Finding the eigenvalues of the Laplacian matrix of a graph_

As we have seen in the previous section (section 2.6.2), finding the eigenvectors of the Laplacian matrix of a graph can solve a partitioning problem. Several methods can be used to find the eigenvectors of a matrix:

– the Lanczos iterative algorithm;

– the Rayleigh quotient iteration method;

– the QR decomposition algorithm;

– the Jacobi iterative algorithm.

All these methods are introduced in [GOL 96].

Among these methods, the Lanczos iterative algorithm is used most often [POT 90, SIM 91, HAG 92a, HEN 95b]. However, the Rayleigh quotient iteration method is also used in some studies [BAR 94, HEN 95c]. In [HEN 95c] it is even favorably coupled with a multilevel method to make it faster.

According to property 2.1 presented in the last section, the Laplacian matrix of a graph is positive semi-definite, so all its eigenvalues are positive. Thus, they can be arranged in an ascending order: 0 ≤ λ1 ≤ λ2 ≤ · · · ≤ λ _n_. There is a trivial eigenvector: the unit vector ( _x_ = (1, . . . , 1) _T_ ). Its eigenvalue is λ = 0; we deduce that λ1 = 0.

The first person to study the properties of these eigenvectors and values is Miroslav Fiedler [FIE 75]. Hence, this family of eigenvectors is often called the Fiedler vectors family. Thus, the first Fiedler vector is the unit vector. The second Fiedler vector is the smallest non-zero eigenvalue of this eigenvalues family. It is the second Fiedler vector that will allow us to find a bisection of the graph. Indeed, if the values of this vector are real, their discretization to {−1; 1} can distribute the vertices of the graph into two sets _V_ 1 and _V_ 2.

Let us try to find a 2 _i_ -partition of _G_. The result found in the case of a graph bisection can be easily generalized. Indeed, the _i_ +1 first eigenvectors of the Fiedler family of the Laplacian matrix of _G_ allow us to find a 2 _i_ -partition of _G_. To do so, it only requires us to discretize each Fiedler vector to {−1; 1}, and then create the corresponding bisections. This process will be studied in more detail in the following subsections (section 2.6.5 and 2.6.6).

## 2.6.4. _Lower bound for constrained graph partitioning_

The study of the spectral properties of the adjacency matrix of the graph allowed us to find a lower bound to the cost cut of a partition of an unweighted graph. The demonstration was made by W. Donath and A. Hoffman in [DON 73].

THEOREM 2.1.– _Let G_ = ( _V, E_ ) _be a graph of unit weight and M Adj its adjacency matrix._

_Let M D be a diagonal matrix such that tr_( _M D_) = ∑i( _M D_) _ii_ = 2| _E_ |.

_Let m_ 1 ≥ _m_ 2 ≥ · · · ≥ _m k_ _be k postive integers such that_ ∑ _i mi_ = | _V_ |.

_Let_ _P k_ = { _V_ 1, . . . , _V k_} _be a partition of V in k parts such that_ ∀ _i_ ∈ {1, . . . , _k_ }, | _V_ 1| = _m i_.

_If we note the eigenvalues of the matrix M D_ − _M Adj by ascending order of value:_ λ1 ≤ λ2 ≤ · · · ≤ λ _k_ ≤ · · · ≤ λ| _V_ |, _then:_

[2.22]

In this theorem, the degree matrix of the graph can be used, instead of _M D_. The eigenvalues used are then the _k_ lowest eigenvalues of the Laplacian matrix of the graph.

In the case of perfectly-balanced partitions, R. Elsässer, T. Lücking and B. Monien [ELS 03], show that when _cut_ ( _P k_) = 1/2 ∑ _i_ | _V i_|λ _i_ , then all the vertices of the graph are adjacent to the cut. The partition has, in that case, a large number of parts, fairly close to the number of vertices. They offer a new lower bound for the partitions that do not fall within this framework.

## 2.6.5. _Spectral methods for contrained partitioning_

The constrained partitioning problem (as defined in Chapter 1) consists of finding a partition _P k_ in _k_ parts that maintains a partitioning balance _bal max_ and with a minimum cut. The spectral methods have been used in many chapters to solve the constrained graph partitioning problem [SIM 91, HEN 93, BAR 94, ALP 95, HEN 95b].

In the previous sections (section 2.6.2 and 2.6.3), we have seen that minimizing the cut of a bisection _P_ 2 = { _V_ 1, _V_ 2} is equivalent to finding the second Fiedler vector of the Laplacian matrix of the graph. The other _P_ Fiedler vectors can be used in ascending order to obtain a 2 _p_ partition. However, as Bruce Hendrickson and Robert Leland point out [HEN 95b], this process cannot be extended endlessly. Only the first three Fiedler vectors (without including the zero eigenvalue vector) provide satisfactory results. We shall now explain this as follows.

Let _G_ = ( _V, E_ ) be a graph and _M Lap_ its Laplacian matrix. Solving the system [2.17] is a linear approach to solve the discrete problem of minimizing the cut. There are therefore two main differences between solving this linear system and finding a solution to the constrained partitioning problem:

– the values of the vector _x_ in equation [2.17] are real and not in {−1; 1};

– solving the linear system [2.17] does not take into account the partition balance.

These are the two approximations we mentioned at the beginning of this section (section 2.6.1).

Thankfully, by associating these two drawbacks we can find a solution to the constrained partitioning problem. Indeed, the discretization of the Fiedler vector, _x_ , to {−1; 1} will maintain the partitioning balance _bal max_. To do so, we have to sort the components of _x_ by ascending values and to discretize the first half of the vector to −1 and the second half to 1. The index _i_ of the vector corresponding to this first half is chosen such that: _x i_ < 0 and this half reaches the maximal weight allowed by the balance; or _x i_ > 0 and this half reaches the minimal weight allowed by the balance. Thus, a graph bisection maintaining the balance has been found.

When several Fiedler vectors are used to find a 2 _p_ -partitioning, each vector is discretized to obtain a bipartition of the graph. But new bisections must maintain the partitioning balance imposed by previous bisections. Because the balance is most difficult to maintain, the number of Fiedler vectors that can be used is limited.

For a more formal and more detailed presentation, the reader may refer to the excellent chapter by Bruce Hendrickson and Robert Leland [HEN 95b].

The processes we have presented only allow us to find partitions in 2, 4, or 8 parts. In order to find a _k_ -partition of the graph, with _k_ an integer strictly greater than 2, the technique of recursive bisection can be used (see Chapter 1, page 18).

The spectral method is a global approach for graph partitioning. It is therefore useful to apply a local optimization algorithm to the partition found [HEN 93]. Such algorithms are called refinement algorithms in the case of graph partitioning. They were introduced earlier in this chapter, section 2.5.

## 2.6.6. _Spectral methods for unconstrained graph partitioning_

Unconstrained graph partitioning requires two cases to be distinguished, depending on whether we seek to minimize one of the two objective functions:

### 2.6.6.1. _Ratio cut_

The minimization of the ratio cut by spectral methods was introduced by [POT 90, HAG 92a, DIN 01]. The first stage of this minimization process consists of, as in the constrained case, solving a linear system. However, linear system [2.21] has to be adapted to the ratio cut. The new linear system to solve is:

[2.23]

Once the Fiedler vectors are found by the methods introduced in subsection 2.6.3, we have to project them onto the partition. It is the stage that differs the most from the constrained case. Indeed, the constraint on the partitioning balance no longer exists. Let _x_ be the second Fiedler vector found, its components are in [−1; 1]. As there is no longer any partitioning balance to be maintained, the projection of x on the bisection _P_ 2 = { _V_ 1, _V_ 2} is very simple: if the component _x i_ of _x_ is negative, the vertex _v i_ will be added to _V_ 1; conversely, if _x i_ > 0 then _v i_ will be added to _V_ 2.

### 2.6.6.2. _Normalized cut_

The normalized cut is widely used to solve image segmentation problems. In this case, the problem is often solved by a spectral method [SHI 98b, SHI 98a, WEI 99, SHI 00, BEN 05]. The normalized cut minimization process is similar, apart from one detail, to the ratio cut process that we have just introduced; in the normalized cut case, the linear system is solved as follows:

[2.24]

## 2.6.7. _Problems and improvements_

One major problem when using spectral methods is that the algorithms used for finding eigenvectors are computationally expensive and memory consumptive. This drawback has been recollected in [HEN 95a], where B. Hendrickson and R. Leland have recommended to use the Lanczos iterative algorithm for graphs of less than 10,000 vertices, and the use of the iterative method combining multilevel and Rayleigh quotient, for larger graphs.

We have estimated the memory used and the computation time required for the Lanczos algorithm and the hybrid multilevel-Rayleigh quotient algorithm. Table 2.1 shows these results. These were obtained from the partitioning tool Chaco on a computer running on GNU/Linux _Debian_ , with an Intel Pentium IV 3 GHz processor and 512 Mo of RAM. This table shows the limits of the Lanczos method, and in general of the non-hybrid spectral methods, both in memory space and in computation time. Indeed, for the tested graphs, with a size greater than 10,000 vertices, the Lanczos algorithm can use up to 360 Mo of memory, as against 44 Mo to the most for the hybrid algorithm. The computation time is also much lower in the case of the hybrid algorithm, with at most 0.54 second of computation, against at most 4.75 seconds for the

**Table 2.1.** _Computation time and memory used by the CHACO program according to the spectral method chosen_

Lanczos algorithm. Besides, the cost obtained with the hybrid method shows that this method can provide results of similar quality to those of the Lanczos algorithm.

This computation time and memory usage explain why the multilevel methods, that allow us solve bigger problems faster, were used often.

# 2.7. Conclusion

This chapter, dedicated to the presentation of the multilevel method, does not pretend to be thorough, or to have described this method with all the nuances present in some classic tools, such as METIS or SCOTCH. However, we have tried to present this method from a rather conventional angle, but also modular, so that the reader could quickly become familiar with this method and adopt some parts according to his needs.

Currently, the most efficient graph partitioning tools all use the multilevel method. These tools are presented in the Appendix. The listing of these tools is followed by a description of the Websites that give archives and benchmarks for constrained graph partitioning. Many graphs and partitions are also available on the book's Website5.

The multilevel method is very efficient; however, it is important to bear in mind that this performance directly depends on the refinement and partitioning algorithms used. The study of these algorithms is therefore essential. On the other hand, the multilevel method is most often used within the constrained partitioning problem context, which is probably bound to change. Indeed, the spectral method, which was gradually replaced by the multilevel method in the constrained case, is still widely used to solve unconstrained graph partitioning problems.

# 2.8. Bibliography

[ALO 85] ALON N., MILMAN V., "λ1, isoperimetric inequalities for graphs, and superconcentrators", _Journal of Combinatorial Theory, series B_ , vol. 38, no. 1, p. 73-88, 1985.

[ALP 95] ALPERT C.J., YAO S.Z., "Spectral partitioning: The more eigenvectors the better", _Proceedings of the ACM/IEEE Design Automation Conference_ , p. 195-200, 1995.

[BAÑ 03] BAÑOS R., GIL C., ORTEGA J., MONTOYA F., "Multilevel heuristic algorithm for graph partitioning", _Proceedings of the European Workshop on Evolutionary Computation in Combinatorial Optimization,_ p. 143-153, 2003.

[BAR 93] BARNARD S.T., SIMON H.D., "A fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems", _Proceedings of the 6th SIAM Conference on Parallel Processing for Scientific Computing_ , p. 711-718, 1993.

[BAR 94] BARNARD S.T., SIMON H.D., "A fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems", _Concurrency: Practice and Experience_ , vol. 6, p. 101-107, 1994.

[BEN 05] BENEZIT F., COUR T., SHI J., "Spectral segmentation with multiscale graph decomposition", _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '05)_ , vol. 2, p. 1124-1131, 2005.

[BEZ 99] BEZRUKOV S., ELSÄSSER R., SCHROEDER U.P., "On bounds for the _k_ -partitioning of graphs", _Computing and Combinatorics: 5th Annual International Conference, COCOON '99, Lecture Notes in Computer Science_ , vol. 1627, p. 154, 1999.

[BUI 89] BUI T., HEIGHAM C., JONES C., LEIGHTON T., "Improving the performance of the Kernighan-Lin and simulated annealing graph bisection algorithms", _Proceedings of the 26th ACM/IEEE Design Automation Conference_ , p. 775-778, 1989.

[BUI 93] BUI T., JONES C., "A heuristic for reducing fill in sparse matrix factorization", _Proceedings of the 6th SIAM Conference on Parallel Processing for Scientific Computing_ , p. 445-452, 1993.

[DHI 04a] DHILLON I.S., GUAN Y., KULLIS B., "Kernel k-means, spectral clustering, and normalized cuts", _Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , p. 551-556, 2004.

[DHI 04b] DHILLON I.S., GUAN Y., KULLIS B., A unified view of kernel k-means, spectral clustering and graph partitioning, Report No. TR-04-25, University of Texas, Austin, 2004.

[DIE 95] DIEKMANN R., MONIEN B., PREIS R., "Using helpful sets to improve graph bisections", _Proceedings of the DIMACS Workshop on Interconnection Networks and Mapping and Scheduling Parallel Computations_ , p. 57-73, 1995.

[DIE 96] DIEKMANN R., LÜLING R., MONIEN B., SPRÄNER C., "Combining helpful sets and parallel simulated annealing for the graph-partitioning problem", _Parallel Algorithms and Applications_ , vol. 8, p. 61-84, 1996.

[DIN 01] DING C., HE X., ZHA H., GU M., SIMON H.D., "A min-max cut algorithm for graph partitioning and data clustering", _Proceedings of the 1st IEEE International Conference on Data Mining_ , p. 107-114, 2001.

[DON 72] DONATH W., HOFFMAN A., "Algorithms for partitioning graphs and computer logic based on eigenvectors of connection matrices", _IBM Technical Disclosure Bulletin_ , vol. 15, no. 3, p. 938-944, 1972.

[DON 73] DONATH W., HOFFMAN A., "Lower bounds for the partitioning of graphs", _IBM Journal of Research and Development_ , p. 420-425, 1973.

[DUT 93] DUTT S., "New faster Kernighan-Lin-type graph-partitioning algorithms", _Proceedings of the IEEE/ACM International Conference on Computer Aided Design (ICCAD)_ , p. 370-377, 1993.

[ELS 03] ELSÄSSER R., LÜCKING T., MONIEN B., "On spectral bounds for the _k_ -partitioning of graphs", _Theory of Computing Systems_ , vol. 36, no. 5, p. 461-478, 2003.

[FID 82] FIDUCCIA C.M., MATTHEYSES R.M., "A linear-time heuristic for improving network partitions", _Proceedings of 19th ACM/IEEE Design Automation Conference_ , p. 175-181, 1982.

[FIE 75] FIEDLER M., "A property of eigenvectors of non-negative symmetric matrices and its application to graph theory", _Czechoslovak Mathematical Journal_ , vol. 25, p. 619-633, 1975.

[GOL 96] GOLUB G., LOAN C.V., _Matrix Computations_ , 3rd edition, Johns Hopkins University Press, Baltimore, 1996.

[GUA 98] GUATTERY S., MILLER G.L., "On the quality of spectral separators", _SIAM Journal on Matrix Analysis and Applications_ , vol. 19, no. 3, p. 701-719, 1998.

[GUP 97] GUPTA A., "Fast and effective algorithms for graph partitioning and sparse matrix ordering", _IBM Journal of Research and Development_ , vol. 41, no. 1/2, p. 171-184, 1997.

[HAG 92a] HAGEN L., KAHNG A.B., "New spectral methods for ratio cut partitioning and clustering", _IEEE Transactions on Computer-Aided Design_ , vol. 11, no. 9, p. 1074-1085, 1992.

[HAG 92b] HAGEN L.W., KAHNG A.B., "A new approach to effective circuit clustering", _Proceedings of the IEEE/ACM International Conference on Computer Aided Design_ , p. 422-427, 1992.

[HAG 97] HAGEN L.W., HUANG D.J.H., Kahng A.B., "On implementation choices for iterative improvement partitioning algorithms", _IEEE Transaction on Computer Aided Design_ , vol. 16, no. 10, p. 1199-1205, 1997.

[HEN 93] HENDRICKSON B., LELAND R., Multidimensional spectral load balancing, Report no. SAND93-0074, Sandia National Laboratories, 1993.

[HEN 95a] HENDRICKSON B., LELAND R., The Chaco User's Guide, Sandia National Laboratories, 2.0 edition, 1995.

[HEN 95b] HENDRICKSON B., LELAND R., "An improved spectral graph partitioning algorithm for mapping parallel computations", _SIAM Journal on Scientific Computing_ , vol. 16, no. 2, p. 452-469, 1995.

[HEN 95c] HENDRICKSON B., LELAND R.W., "A multilevel algorithm for partitioning graphs", _Proceedings of Supercomputing_ , 1995.

[HU 98] HU Y.F., BLAKE R.J., EMERSON D.R., "An optimal migration algorithm for dynamic load balancing", _Concurrency-Practice and Experience_ , vol. 10, no. 6, p. 467-483, 1998.

[KAR 95] KARYPIS G., KUMAR V., "Analysis of multilevel graph partitioning", _Proceedings of Supercomputing_ , 1995.

[KAR 98a] KARYPIS G., KUMAR V., "A fast and high quality multilevel scheme for partitioning irregular graphs", _SIAM Journal of Scientific Computing_ , vol. 20, no. 1, p. 359-392, 1998.

[KAR 98b] KARYPIS G., KUMAR V., METIS: A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices, 4.0 edition, University of Minnesota, September 1998.

[KAR 98c] KARYPIS G., KUMAR V., "Multilevel algorithms for multiconstraint graph partitioning", _Proceedings of Supercomputing_ , p. 1-13, 1998.

[KAR 98d] KARYPIS G., KUMAR V., "Multilevel k-way partitioning scheme for irregular graphs", _Journal of Parallel and Distributed Computing_ , vol. 48, no. 1, p. 96-129, 1998.

[KER 70] KERNIGHAN B.W., LIN S., "An efficient heuristic procedure for partitioning graphs", _Bell System Technical Journal_ , vol. 49, no. 2, p. 291-307, 1970.

[KRI 84] KRISHNAMURTHY B., "An improved min-cut algorithm for partitioning VLSI networks", _IEEE Transactions on Computers_ , vol. 33, no. 5, p. 438-446, 1984.

[MAR 96] MARTIN O.C., OTTO S.W., "Combining simulated annealing with local search heuristics", _Annals of Operations Research_ , vol. 63, p. 57-75, 1996.

[MEY 05] MEYERHENKE H., SCHAMBERGER S., "Balancing parallel adaptive FEM computations by systems of linear equations", _European Conference on Parallel Processing, Lecture Notes in Computer Science_ , vol. 3648, 2005.

[MOH 91] MOHAR B., "The laplacian spectrum of graphs", in ALAVI E., CHARTRAND G., OELLERMANN O., SCHWENK A. (eds), _Graph Theory, Combinatorics, and Applications_ , John Wiley & Sons, 1991.

[MOH 97] MOHAR B., "Some applications of Laplace eigenvalues of graphs", Graph Symmetry: Algebraic Methods and Applications, NATO ASI Series C, vol. 497 1997.

[MON 04] MONIEN B., SCHAMBERGER S., "Graph partitioning with the party library: helpfulsetsin practice", _Proceedings of the Symposium on Computer Architecture and High Performance Computing_ , p. 198-205, 2004.

[PAP 07] PAPA D., MARKOV I., "Hypergraph partitioning and clustering", in GONZALEZ T. (ed.), _Approximation Algorithms and Metaheuristics_ , CRC Press, 2007.

[PEL 96] PELLEGRINI F., ROMAN J., Experimental analysis of the dual recursive bipartitioning algorithm for static mapping, Report No. 1038-96, LaBRI, University of Bordeaux I, 1996.

[PEL 07] PELLEGRINI F., "A parallelisable multilevel banded diffusion scheme for computing balanced partitions with smooth boundaries", _European Conference on Parallel Processing, Lecture Notes in Computer Science_ , vol. 4641, 2007.

[POT 90] POTHEN A., SIMON H.D., LIOU K.P., "Partitioning sparse matrices with eigenvectors of graphs", _SIAM Journal on Matrix Analysis and Applications_ , vol. 11, no. 3, p. 430-452, 1990.

[RAN 01] RANTAKOKKO J., "A local refinement algorithm for data partitioning", _Proceedings of the 5th International Workshop on Applied Parallel Computing (2000), Lecture Notes in Computer Science_ , vol. 1947, p. 140-148, 2001.

[SAN 89] SANCHIS L.A., "Multiple-way network partitioning", IEEE Transactions on Computers, vol. 38, no. 1, p. 62-81, 1989.

[SAN 93] SANCHIS L.A., "Multiple-way network partitioning with different cost functions", _IEEE Transactions on Computers_ , vol. 42, no. 12, p. 1500-1504, 1993.

[SCH 03] SCHAMBERGER S., "Improvements to the helpful-set heuristic and a new evaluation scheme for graphs-partitioners", _Proceedings of the International Conference on Computational Science and its Applications_ , p. 49-59, 2003.

[SHI 98a] SHI J., BELONGIE S., LEUNG T., MALIK J., "Image and video segmentation: The normalized cut framework", _Proceedings of the IEEE International Conference on Image Processing_ , p. 943-947, 1998.

[SHI 98b] SHI J., MALIK J., "Motion segmentation and tracking using normalized cuts", _Proceedings of the 6th IEEE International Conference on Computer Vision (ICCV)_ , p. 1154-1160, 1998.

[SHI 00] SHI J., MALIK J., "Normalized cuts and image segmentation", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 22, no. 8, p. 888-905, 2000.

[SIM 91] SIMON H.D., "Partitioning of unstructured problems for parallel processing", _Computing Systems in Engineering_ , vol. 2, p. 135-148, 1991.

[SON 94] SONG J., "A partially asynchronous and iterative algorithm for distributed load balancing", _Parallel Computing_ , vol. 20, no. 6, p. 853-868, 1994.

[WAL 95] WALSHAW C., CROSS M., EVERETT M.G., "A localized algorithm for optimizing unstructured mesh partitions", _International Journal of Supercomputer Applications_ , vol. 9, no. 4, p. 280-295, 1995.

[WAL 97] WALSHAW C., CROSS M., EVERETT M.G., "Parallel dynamic graph partitioning for adaptive unstructured meshes", _Journal of Parallel and Distributed Computing_ , vol. 47, no. 2, p. 102-108, 1997.

[WAL 00a] WALSHAW C., CROSS M., "Mesh partitioning: a multilevel balancing and refinement algorithm", _SIAM Journal on Scientific Computing_ , vol. 22, p. 63-80, 2000.

[WAL 00b] WALSHAW C., CROSS M., MCMANUS K., "Multiphase mesh partitioning", _Applied Mathematical Modeling_ , vol. 25, p. 123-140, 2000.

[WEI 99] WEISS Y., "Segmentation using eigenvectors: A unifying view", _Proceedings of the 7th IEEE International Conference on Computer Vision_ , p. 975-982, 1999.

[ZHA 02] ZHAO Z., TAO L., ZHAO Y., "An effective algorithm for multiway hypergraph partitioning", _IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications_ , vol. 49, no. 8, p. 179-1092, 2002.

1 Chapter written by Charles-Edmond Bichot.

1 Chapters 6 and  describe the adaptation of many metaheuristics, from the most popular (genetic algorithms, simulated annealing) to the lesser known (fusion-fission), to the graph partitioning problem.

2 Different graph partitioning tools, available on the Internet, such as METIS, SCOTCH and JOSTLE, are presented in the Appendix.

3 In an undirected graph _G_ = ( _V, E_ ), the degree of a vertex _v_ ∈ _V_ is the number of edges to which this vertex belongs: _deg_ ( _v_ ) = |{( _V_ , _v_ ′) ∈ _E_ , _v_ ′ ∈ _v_ }|.

4 A graph separator is a subset of vertices the removal of which disconnects the graph.

5The URL of the book's Website, "Graph partitioning: optimization and applications", is: perso.ec-lyon.fr/charles-edmond.bichot/livre_ partitionnement.

# Chapter 3

# Hypergraph Partitioning 1

A graph is a data structure that can express connections between two vertices. However, in reality many problems have connections defined on a set of vertices, whose cardinality is not necessarily two. To handle these cases, the graph structure has been generalized to form an entity called a hypergraph, which will be the subject of this chapter.

This chapter aims to provide the reader with a slightly different perspective on the classic use of graphs for partitioning problems defined from topological data.

# 3.1. Definitions and metrics

This sections aims to provide formal definitions of the hypergraph object and the hypergraph partitioning problem. Subsequently, the word "graph" will refer to a simple undirected graph.

## 3.1.1. _Hypergraph and partitioning_

DEFINITION 3.1. (HYPERGRAPH)– _A hypergraph H_ = ( _V_ ,  ) _is a structure made up of a set V of elements called vertices, and a set  of subsets of V called hyperedges._

_V_ ( _H_ ) _and_  ( _H_ ) _respectively represent the set of vertices and the set of hyperedges of H . A hypergraph example can be seen inFigure 3.1(a)._

**Figure 3.1.** _Two representations of a hypergraph_

A hypergraph can also be defined as a triplet ( _V_ ,  , _P_ ), where _V_ is the set of vertices,   the set of hyperedge labels, and _P_ the set of edges connecting an element of _V_ to an element of  . The elements of _P_ are called _pins_ by analogy with the electronics terminology. A hyperedge (sometimes called _net_ that is even now borrowed from the vocabulary of electronics) is therefore defined as the set of pins connected to one label. This representation corresponds to the example in Figure 3.1(b).

From this definition, we can make several observations:

– a graph is a hypergraph, where all the hyperedges are subsets of _V_ with a cardinality of 2;

– a hypergraph _H_ = ( _V_ ,  , _P_ ) is an isomorphic of a bipartite graph _G_ = ( _V_ ,  , _P_ ). However, the bipartite graph structure does not consider the existence of the hyperedge concept, which structures the set of the elements of _P_.

It is also possible to define weights for both the vertices and the hyperedges. Usually, all the pins of the same hyperedge share the same weight, though there are models in which the weight of each pin can be different.

From these similarities, it is possible to generalize the partitioning problem into hypergraphs, in a similar way as defined for graphs.

DEFINITION 3.2. (HYPERGRAPH _k_ -PARTITIONING PROBLEM)– _The k-partitioning problem of a hypergraph H_ = ( _V_ ,  ), _whose hyperedges and vertices are weighted in_  +, _and where w V_ _V_ →  + _and w_  :   →  + _are the applications that associate each vertex or hyperedge with its weight, can be formulated as follows:_

_Find a family_ ( _V i_)1≤ _i_ ≤ _k_ _of subsets of V such that:_

_1) the weight of each part Vi is the same, which means that it is necessary to minimize the difference of weights between each pair of parts:_

[3.1]

_2) the cut_  _is minimal, where I is the set of border hyperedges, i.e. hyperedges connecting at least two vertices from distinct parts._

It is also possible to define a "vertices" hypergraph partitioning, i.e. when borders between parts are defined by vertices and not by hyperedges. Since the latter is rarely used, we will simply refer by hypergraph partitioning when separation between parts are defined by hyperedges.

## 3.1.2. _Metrics for hypergraph partitioning_

As seen previously, a hyperedge is generally made up of more than one pin. This means that the cut metrics are not suitable for solving the problem, because they relate to the number of hyperedges on the border and not to the connection of the different parts, since a hyperedge can have its ends in more than two areas. In particular, when hyperedges are used to model volumes of data to exchange, it is interesting to know how many times the exchange of these data has to be done, in other words, how many parts are adjacent through a given hyperedge.

A popular metric [DEV 02] to solve this problem is the so-called λ − 1 metric. If a hyperedge connects λ parts, for each vertex of this hyperedge, information has to be communicated (λ − 1) times. From this observation, we can define the cut as ∑ _e_ ∈ _I_ (λ _e_ − 1) · _w_  ( _e_ ). It is notable that in the case of a graph, λ ≤ 2, this expression of the cut is the classic formulation.

# 3.2. Connections between graphs, hypergraphs, and matrices

We have previously seen that there is a close relationship between the graph and the hypergraph concepts.

In addition, a graph can be represented by its adjacency matrix, and, similarly, the structure of a structurally symmetric square matrix1 can be modeled by an undirected graph.

A hypergraph _H_ = ( _V_ ,  , _P_ ) can be modeled by a matrix _M_ , so that the columns of the matrix correspond to the elements of _V_ , the rows to the elements of  , and the non-zero elements to the pins _P_. Inversely, with each matrix _M_ we can associate a hypergraph   , and it is called the _row-net_ representation of the matrix [ÇAT 99]. The hypergraph in Figure 3.1 is the _row-net_ representation of the following matrix:

**Figure 3.2.** _Example of a partition of a hypergraph and the corresponding graph_

Similarly, by reversing the notions of rows and columns, it is possible to define a _column-net_ representation of the hypergraph _H_ , where the rows of the matrix _M_ correspond with the vertices of _H_.

Hypergraphs are therefore widely used in sparse linear algebra problems which do not always involve structurally symmetric matrices (see Chapter 6 where the chapter describes several methods for solving linear systems). In addition, it is common to use this matrix representation to describe hypergraphs. In the remainder of this chapter, we will frequently use this matrix representation.

Another advantage of this matrix representation is the ability to compare directly the properties of graphs with those of hypergraphs. Indeed, a structurally symmetric square matrix _M_ can be modeled by a simple undirected graph _G_ or by a hypergraph _H_. Then the fundamental difference is that the edges of _G_ are the non-zero elements of _M_ , whereas the hyperedges of _H_ are the rows of _M_ .

# 3.3. Algorithms for hypergraph partitioning

As graph partitioning problems have been shown to be NP-Hard [GAR 79], it is not a surprise that their extension to hypergraph partitioning problems is also NP-Hard [LEN 90].

The structural similarities between graphs and hypergraphs have resulted in the selection of the same kind of algorithms for hypergraph problems as for graph ones.

The most popular method is therefore based on a multilevel algorithm2, for both sequential tools [ÇAT, KAR 98] and parallel tools3 [DEV 06b, TRI 04]. The principle remains to reduce the size of the problem by successive stages (coarsening), to solve the coarsened problem (initial partitioning), and then to project and optimize this solution upto the initial problem (uncoarsening and refinement). However, some phases of the method require specific algorithms in case of hypergraphs.

## 3.3.1. _Coarsening_

This phase aims to reduce the size of the problem by successively decreasing the number of vertices.

However, the problem of the choice of grouping together the vertices is more complex in the case of a hypergraph, because a hyperedge connects a set of vertices and not a pair of vertices.

The most popular method for choosing the sets (or, most often, only pairs) of vertices to merge with the graph coarsening phase is the HEM method ( _heavy edge matching_ ), which consists in selecting the pairs of vertices connected by the edge of maximal weight. This method is in fact based on the hypothesis that if two vertices are connected by an edge of significant weight they are more likely to share the same property (belonging to the same part of the partitioning problem). This simple idea usually works well on easy problems, however it can be occasionally defective on harder instances [CHE 09]. In this report, the concept of algebraic distance is presented and its use, in a multilevel algorithms context, seems to be preferable as compared to use of more primitive metrics. For example, classical matching methods have the drawback of depending only on the properties of the incident edges. However, in practice, of using the weight of the edges as a measure of the topological proximity of the vertices seems sufficient in most of the cases to ensure the smooth functioning of the multilevel methods.

In the context of hypergraphs, there is no direct way to estimate this topological proximity, even if this concept remains, _a priori_ , very close to the concept of the graphs. Indeed, we will try to define a binary relation, a measure of similarity between two vertices, which is obvious in a simple graph when the edge (if it exists) between these two vertices is used. For hypergraphs, the number of hyperedges common to two vertices seems to be a good choice for such a measure.

The _row-net_ model of a hypergraph _H_ by a (sparse) matrix _M_ , where columns are the vertices of _H_ and rows the hyperedges of _H_ , and where non-zero elements are the pins, allows us to visualize the measure of similarity between two vertices. Indeed, we aim to gather the columns whose structure is similar. In linear algebra, the inner product of two columns is considered to be a good metric to characterize this property, because when the coefficients are Boolean, it corresponds to the number of hyperedges common to both vertices.

In the case of a hypergraph with weighted hyperedges, the coefficients of the matrix can be defined as the value of the weight of the hyperedge, divided by the number of pins it contains. This normalization is essential, because the hyperedges can have various sizes and it is interesting to avoid favoring those with the most pins.

Most algorithms and software [ÇAT 99, KAR 97, VAS 05, DEV 06b] use an inner product matching (Inner Product Matching or Heavy Connectivity Matching).

* * *

**Algorithm 3.1.** Inner product matching computation

* * *

* * *

However, we should note that the calculation of the inner product for every pair of columns is equivalent to the calculation of the matrix product _M t_ · _M_ , which is potentially very expensive in memory space as well as computation time. This explains why most of the practical implantations do not calculate every possible pair and use a greedy inner product matching algorithm [ÇAT 99, DEV 06b]. Algorithm 3.1 is one such algorithm. We can point out that the algorithm does not form the matrix _M t_ · _M_ by merely calculating the useful inner products, where the vertices already matched are ignored and the temporary results deleted. This strategy can also be transposed to the case of a hypergraph distributed in memory and it is particularly advantageous because it can avoid the transfer of partial inner products between processors.

The matched vertices (or the sets of matched vertices), are then merged, as in the graph coarsening case. The hyperedges with arity of 1 are ignored because they have no impact on the cut during partitioning. The hyperedges sharing the same set of pins are merged, as the vertices. A consequence of this process is that the coarsening is more significant on the vertices than on the hyperedges, because their number decreases at each stage of the coarsening, while the number of the hyperedges remains constant. However, the global size of the hypergraph also decreases as the number of pins decreases.

However, there are other processes of coarsening [BUL 08], such as those resulting from multigrid algorithms. The latter can also reduce both dimensions of a matrix, i.e. in the case of a hypergraph they can coarsen the vertices and the hyperedges simultaneously. But, this approach is not currently used for practical purposes, because it has the disadvantage of rapidly forming very dense hypergraphs, often too dense to be efficiently processed by initial partitioning algorithms.

## 3.3.2. _Initial partitioning and uncoarsening and refinement_

The initial partitioning phase for hypergraphs uses the same types of algorithms as those described for graphs. Most software implantations use greedy algorithms only, as the exact solutions (obtained by an exhaustive computation of the possible solutions) do not result in significant gains [ÇAT 99, KAR 97, VAS 05, DEV 06b].

The low interest in exact methods for the initial partitioning can be explained by the fact that the coarsening phase induces a loss of topological information, which is more significant than the gain in quality, obtained by using an exact method.

In addition, it is relevant to use heuristics to stop the coarsening on a larger hypergraph rather than a brute force algorithm on a smaller one, since its complexity is exponential depending on the number of vertices.

It is notable that spectral methods can also be used, either directly or by symmetrizing the hypergraph ( _M t_ · _M_ ) to obtain a graph.

## 3.3.3. _Uncoarsening and refinement_

The solutions used in the uncoarsening and refinement of the partition during the growing phase also result from graph algorithms, the most popular being Kernighan-Lin (KL) [KER 70] and Fiduccia-Mattheyses (FM) [FID 82]. However, because their generalization in the case of _k_ -partitioning is rather complex to describe and to implement, many implementations use a recursive bipartitioning technique. In this context, it is possible to slightly alter FM to be able to keep, for parallel implementations, some of its interesting characteristics, in particular its hill-climbing capacities. For example, Zoltan [DEV 06b] uses, within a distributed memory context, an algorithm where each pass consists of two stages, each of them being the shifting of vertices to the target partition, which changes between the two stages. This process is used to avoid conflicting shiftings when several processors are used. Even if it seems more restrictive than the classic version of FM, we must recollect that the hyperedges can connect more than two vertices, and therefore to alter the cut related to a hyperedge, it is necessary to shift its vertices into the same part. Indeed, in the case of a graph, since an edge has only two ends, the shifting of one of them4 to another part is always sufficient to alter the cut related to this edge.

In the same way, as for the graphs, other refinement approaches are possible, in particular, evolutionary algorithms such as genetic algorithms. However, they are currently used in research codes only, since the approaches previously described are preferred for production tools.

It is possible to use methods resulting from multigrid aglorithms by considering the hypergraph as a matrix. These relaxation methods can have a wider impact than pure combinatorial algorithms previously presented.

However, as for graph partitioning, most of the available software implantations are based on a combinatorial approach for multilevel algorithms, and therefore use coarsening with a matching based on inner products calculation to measure the proximity between two vertices, a greedy initial partitioning algorithm, and an uncoarsening based on a direct projection of the partitioning and an FM refinement.

# 3.4. Purpose

## 3.4.1. _Hypergraph partitioning benefits_

It has been proved that for many problems, the partitioning model that uses a graph does not correspond to an exact model [HEN 98, HEN 00b].

For example, consider the case of the grid in Figure 3.3, where each vertex requires information of the neighboring vertices. We try to calculate a bisection of the grid in two regions _P_ 0 and _P_ 1 which will each be allocated to a different processor. The aim is to reduce the volume of data to be exchanged.

**Figure 3.3.** _Communication cost associated with the vertex A during the bipartitioning of a grid with a graph and hypergraph model_

In the case of a bipartitioning that uses a graph model (Figure 3.3(a)), since the vertex _A_ of the part _P_ 1 has 2 neighbors in the part _P_ 0, the number of required communications associated with this vertex _A_ is 2. However, we can notice that both vertices _B_ and _C_ are in the same part, therefore it is possible that a single communication of the data from A provides sufficient information for these two vertices.

If we model the problem with a hypergraph whose hyperedges correspond to the connection "are neighbors", we notice that _A_ , _B_ and _C_ are on the same hyperedge, and therefore the number of required communications is only 1 (Figure 3.3(b)). We notice that, in this simple example, the hypergraph model is an accurate model of the problem, whereas the simple graph model is merely an approximation.

It is, however, interesting to note that, for practical purposes, the results from both models can be very similar, particularly in the case of regular problems [HEN 00b, VAS 05, BIL 99].

## 3.4.2. _Matrix partitioning_

One of the most popular applications [HEN 00b, HEN 00a, VAS 05, ÇAT 99] of hypergraph partitioning is the partitioning of sparse matrices, specially to improve matrix-vector products ( _SpMV_ ) efficiency, mostly by decreasing communication volumes.

We have previously seen that a hypergraph can represent an ordinary sparse matrix. In the case of a _column-net_ representation ( _row-net_ respectively), a hypergraph partitioning corresponds to the partitioning of the rows (and respective columns) of the matrix. These two models are commonly called 1D partitioning of the matrix, because only one of the 2D is used.

**Figure 3.4.** _Example of a sparse matrix 2D partitioning_

In the case of a 1D model, if the input and output vectors are partitioned in a way that is compatible with the matrix, only one stage of communication is involved during the computation of the matrix-vector product: during the application of the input vector in the case of the _column-net_ model, and during the accumulation in the output vector with the _row-net_ model.

There are other models, known as 2D partitioning, that allow us to directly work on the non-zero elements of the matrix. Figure 3.4 shows such a partitioning. In that case, communications can occur during both stages of the multiplication computation; however, their total volume can be lower than the one obtained from a 1D partitioning.

Several models allow us to create a 2D partitioning, but we will mainly distinguish two types of approach.

The first method, known as _coarse-grain_ [ÇAT 01b], consists of partitioning twice, each time in 1D. The first one aims to partition the rows of the matrix into a balanced distribution of non-zero elements, and the second one has to simultaneously distribute the non-zero elements of all the sets of rows by using a multicriteria optimization.

A second method, known as _fine-grain_ [ÇAT 01a], consists of partitioning the hypergraph whose vertices are the non-zero elements of the matrix and whose hyperedges are defined by the rows and the columns of the matrix. This approach usually produces the best results, at the cost of a generally long computation time, since the number of vertices in the hypergraph is equal to the number of non-zero elements in the matrix, whereas it is equal to the number of rows (or columns) for the other methods. Intermediate methods have, however, emerged [VAS 05, BOM 08], which try to combine the speed of a 1D partitioning with the quality of a 2D partitioning. To do so, they generally use a 1D partitioning then a " _fine-grain_ " partitioning on the borders and their close neighbors, in order to optimize the partitioning.

**Figure 3.5.** _Comparison of the cut during partitioning into 64 parts with graph or hypergraph models on various irregular problems. The numbers on the right are the numbers of processors used during the test to calculate the partitioning. The results are averages out of 20 executions, then they are normalized with regard to Zoltan on a processor_

**Table 3.1.** _Matrices (symmetric) and test hypergraphs_

## 3.4.3. _Practical results_

Figure 3.5 presents the cut with the λ − 1 metric (so the exact communication volume, in a matrix-vector product context), on the matrices presented in Table 3.1. The hypergraph model is used by ZOLTAN [DEV 02] and PARKWAY [TRI 04], while the graph model is used for experiments with PARMETIS. Note that test matrices are symmetric, so usually it is a typical use case for graph partitioning.

It appears that the use of hypergraphs proves to be more interesting for certain types of problems, in particular, irregular ones. The results of both hypergraph partitioning tools are very close and the cost associated with the partition obtained is much lesser than the cost resulting from the graph partitioning with Parmetis. In addition, in some instances such as Xyce680s, it appears that the partitioning calculation by itself is faster with a hypergraph model, than with a graph model.

However, for regular problems, such as those resulting from mesh partitioning, the number of neighboring vertices is generally constant (or almost constant), which implies the existence of a proportionality connection between the cut of the graph and the cut of the hypergraph, and consequently the choice of the model is less significant.

## 3.4.4. _Repartitioning_

Another problem that can benefit by using a better model is the repartitioning model, which is the core of dynamic load and data balancing. The data are initially partitioned into _k_ parts and the problem consists not only in (re-)partitioning them into _k_ equal parts while keeping the communications at a minimum, but also in limiting the cost of migration of the data with respect to their initial partitioning. These techniques are essentially used during numerical simulations transfer iterations alter the connections between the data, in other words, alter the topology.

The traditional approach to solve this problem is to add _k_ anchor vertices (one per part) which are fixed and connected with all the vertices initially belonging to this part. In the case of a hypergraph model, we can only add a single hyperedge for each part, and the cut calculated during partitioning is an exact model of the redistribution cost [DEV 02, DEV 06b].

## 3.4.5. _Use of hypergraphs within a mesh partitioning context_

Another possible application of hypergraph partitioning relates to mesh partitioning. A mesh can normally be viewed as a structure dealing with nodes and elements. From a combinatorial point of view, it is possible to model a mesh as a bipartite graph, having the elements and the nodes in the form of vertices. The mesh partitioning problem generally consists of forming a partition of the set of elements by minimizing the dependencies between subsets.

When the dependencies are the nodes adjacent to the elements, a hypergraph model of mesh partitioning can be defined: the vertices of the hypergraph are the elements and the hyperedges the nodes of the mesh (Figure 3.6). We can solve accurately the mesh partitioning problem by calculating the hypergraph partitioning.

## 3.4.6. _Other applications_

The choice of a "hypergraph" model can express more constraints than a "graph" model, and they can be used to solve diverse problems more efficiently [ÇAT 07, DEV 06a].

Although the algorithms used are generally quite similar for graphs and hypergraphs, use of hypergraphs can often extend these solutions to new instance

**Figure 3.6.** _Hypergraph model of mesh partitioning. The vertices of the hypergraph (black squares) are the elements of the mesh, and the nodes of the mesh are the hyperedges (gray discs)_

**Figure 3.7.** _Singly-bordered block diagonal form of a matrix after a 4-partitioning of the rows. The darker blocks correspond to the columns having non-zero elements in rows that are allocated to different parts. The zone in white includes only zero elements_

classes. Thus, for the sparse matrices renumbering problem, the "nested dissections" technique5, which is very popular in the case of square symmetric matrices, can be generalized by the HUND (hypergraph unsymmetric nested dissection) algorithm [GRI 08] in particular for dealing with ordinary matrices while using a hypergraph. The principle is to partition the rows of the matrix using a hypergraph ( _column-net_ model). The columns of the matrix are then reordered according to the way they connect rows that are allocated to different parts. Figure 3.7 shows this process and the decomposition in _singly-bordered block-diagonal form_. In the same way as decomposition by nested dissections for a symmetric factorization, the filling can only occur within the blocks thus formed.

# 3.5. Conclusion

Hypergraphs are very useful tools for some applications by extending the possibilities of graph models.

However, as they are only an extension of graphs, the algorithms used for their partitioning are similar to those used for graph partitioning, with often the same restrictions (scalability in time within a distributed and massively parallel context).

The main point to notice is that for irregular problems, the hypergraph model often leads to better results, mainly by allowing an exact model of the problem, which is impossible to obtain from graphs.

# 3.6. Software references

There are several software in the public domain that can be used to compute hypergraph partitioning.

For sequential forms, HMETIS [KAR 98] and PATOH [ÇAT] are the most wellknown software, and for parallel forms, Parkway [TRI 04] and ZOLTAN [DEV 02, DEV 09] are most widely used.

However, it should be noted that some of the graph partitioning software, such as Scotch [PEL 04], provide, for some features, a way to work on bipartite graphs that are isomorphic to hypergraphs.

# 3.7. Bibliography

[BIL 99] BILDERBACK M., "Improving unstructured grid application execution times by balancing the edge-cuts among partitions", _SIAM Conference on Parallel Processing for Scientific Computing_ , 1999.

[BOM 08] BOMAN E.G., WOLF M.M., A nested dissection approach to sparse matrix partitioning for parallel computations, Report No. 2008-5482J, Sandia National Laboratories, NM, 2008.

[BUL 08] BULUÇ A., BOMAN E.G., "Towards scalable parallel hypergraph partitioning", _CSRI Proceedings 2008_ , Sandia National Laboratories, p. 109-119, 2008.

ÇAT] ÇATALYÜREK Ü.V., AYKANAT C., PATOH: A multilevel hypergraph partitioning tool,Version 3.0, Department of Computer Engineering, Bilkent University, Ankara. PaToH is available at [bmi.osu.edu/˜umit/software.htm.

[ÇAT 99] ÇATALYÜREK Ü.V., AYKANAT C., "Hypergraph-partitioning based decomposition for parallel sparse-matrix vector multiplication", _IEEE Transactions on Parallel and Distributed Systems_ , vol. 10, no. 7, p. 673-693, 1999.

[ÇAT 01a] ÇATALYÜREK Ü., AYKANAT C., "A Fine-grain hypergraph model for 2D decomposition of sparse matrices", _Proceedings of IPDPS 8th International Workshop on Solving Irregularly Structured Problems in Parallel (Irregular 2001)_ , April 2001.

[ÇAT 01b] ÇATALYÜREK Ü., AYKANAT C., "A Hypergraph-partitioning approach for coarsegrain decomposition", _Proceedings of Supercomputing 2001_ , ACM, 2001.

[ÇAT 07] ÇATALYÜREK Ü.V., BOMAN E.G., DEVINE K.D., BOZDA  D., HEAPHY R., FISK L.A., "Hypergraph-based dynamic load balancing for adaptive scientific computations", _Proceedings of 21st International Parallel and Distributed Processing Symposium (IPDPS)_ , IEEE, 2007.

[CHE 09] CHEN J., SAFRO I., Algebraic distance on graphs, Report, Argonne National Laboratory, 2009.

[DEV 02] DEVINE K., BOMAN E., HEAPHY R., HENDRICKSON B., VAUGHAN C., "Zoltan data management services for parallel dynamic applications", _Computing in Science and Engineering_ , vol. 4, no. 2, p. 90-97, 2002.

[DEV 06a] DEVINE K.D., BOMAN E.G., KARYPIS G., "Partitioning and load balancing for emerging parallel applications and architectures", in HEROUX M., RAGHAVAN A., SIMON H. (eds), Frontiers of Scientific Computing, SIAM, Philadelphia, 2006.

[DEV 06b] DEVINE K., BOMAN E., HEAPHY R., BISSELING R., ÇATALYÜREK U., "Parallel hypergraph partitioning for scientific computing", Proceedings of 20th International Parallel and Distributed Processing Symposium (IPDPS'06), IEEE, 2006.

[DEV 09] DEVINE K.D., BOMAN E.G., RIESEN L.A., ÇATALYÜREK Ü.V., CHEVALIER C., "Getting started with zoltan: A short tutorial", in NAUMANN U., SCHENK O., SIMON H.D., TOLEDO S. (eds), _Combinatorial Scientific Computing_ , No. 09061, Dagstuhl Seminar Proceedings, Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl, 2009.

[FID 82] FIDUCCIA C.M., MATTHEYSES R.M., "A linear-time heuristic for improving network partitions", _Proceedings of the 19th ACM/IEEE Design Automation Conference_ , p. 175-181, 1982.

[GAR 79] GAREY M.R., JOHNSON D.S., _Computers and Intractability: A Guide to the Theory of NP–Completeness_ , W.H. Freeman, San Francisco, 1979.

[GRI 08] GRIGORI L., BOMAN E., DONFACK S., DAVIS T., _Hypergraph unsymmetric nested dissection_ ordering for sparse LU factorization, Report No. 2008-1290J, Sandia National Labs, Submitted to _SIAM Journal of Scientific Computing_ , 2008.

[HEN 98] HENDRICKSON B., "Graph partitioning and parallel solvers: Has the Emperor no clothes?", Lecture Notes in Computer Science, vol. 1457, p. 218-225, 1998.

[HEN 00a] HENDRICKSON B., KOLDA T.G., "Graph partitioning models for parallel computing", _Parallel Computing_ , vol. 26, p. 1519-1534, 2000.

[HEN 00b] HENDRICKSON B., KOLDA T.G., "Partitioning rectangular and structurally nonsymmetric sparse matrices for parallel processing", _SIAM Journal on Scientific Computing_ , vol. 21, no. 6, p. 2048-2072, 2000.

[KAR 97] KARYPIS G., AGGARWAL R., KUMAR V., SHEKHAR S., "Multilevel hypergraph partitioning: Applications in VLSI domain", _Proceedings of the 34th Design Automation Conference_ , ACM, p. 526-529, 1997.

[KAR 98] KARYPIS G., KUMAR V., AGGARWAL R., SHEKHAR S., HMETIS, A Hypergraph partitioning package version 1.0.1, University of Minnesota, Department of Computer Science and Engineering, Army HPC Research Center, Minneapolis, 1998.

[KER 70] KERNIGHAN B.W., LIN S., "An efficient heuristic procedure for partitioning graphs", _The Bell System Technical Journal_ , vol. 49, no. 2, p. 291-307, 1970.

[LEN 90] LENGAUER T., _Combinatorial Algorithms for Integrated Circuit Layout_ , John Wiley & Sons, New York, 1990.

[PEL 04] PELLEGRINI F., GOUDIN D., "Using the native mesh partitioning capabilities of SCOTCH 4.0 in a parallel industrial electromagnetics code", _11th SIAM Conference on Parallel Processing for Scientific Computing_ , February 2004.

[TRI 04] TRIFUNOVIC A., KNOTTENBELT W.J., "Parkway 2.0: A parallel multilevel hypergraph partitioning tool", _Proceedings of 19th International Symposium on Computer and Information Sciences (ISCIS 2004)_ , vol. 3280 of Lecture Notes in Computer Science, Springer, p. 789-800, 2004.

[VAS 05] VASTENHOUW B., BISSELING R.H., "A two-dimensional data distribution method for parallel sparse matrix-vector multiplication", _SIAM Review_ , vol. 47, no. 1, p. 67-95, 2005.

* * *

1 Chapter written by Cédric CHEVALIER.

1 A square matrix _M_ = ( _m ij_)( _i,j_ )∈{1...n}2 is structurally symmetric if and only if ∀( _i, j_ ) ∈ {1 . . . _n_ }2, _m ij_ ≠ 0 ⇒ _m ji_ ≠ 0.

2 The multilevel method for graph partitioning is presented in Chapter 2.

3 The parallelization of graph partitioning algorithms is presented in Chapter 4.

4 Refinement algorithms applied to vertices of the border.

5 See Chapter 6 on the use of graph partitioning for sparse linear calculation.

# Chapter 4

# Parallelization of Graph Partitioning 1

This chapter provides a state-of-the-art treatise of the different techniques implemented by many authors in order to produce in-parallel where the quality of these partitions matches the best-known sequential algorithms. It focuses particularly on the parallelization of the multilevel framework, which is at the core of many partitioning tools, by looking at the parallel algorithms required by the phases of graph partitioning, and local refinement of the produced partitions, which are the key issues.

# 4.1. Introduction

## 4.1.1. _Need for parallelism_

Parallelization of graph partitioning methods is influenced by three factors of mixed importance. The one that initially first comes to the mind is the computing acceleration by increasing the number of processing elements allocated to the resolution of the problem1. However, it is not the most important one, since in general the partitioning time represents only a fraction of the processing time of scientific applications using partitioning computed by this way. The fundamental criterion is actually the memory space: the size of the graphs that have to be processed keeps on increasing, and for some of the applications it now exceeds a billion vertices, thus making their storage impossible on a single computer. The third factor also results from the size of the graphs, and is due to the input/output bandwidth of the parallel machines in the case of dynamic repartitioning: it is not possible to delegate the task for repartitioning the communication graph of a parallel simulation to a sequential machine and then propagate back the partitioning result to the set of computing nodes; all this should be done directly in parallel.

## 4.1.2. _Multilevel framework_

Experience has shown that, for many types of graphs used in scientific computing2, the best partitioning quality is obtained using a multilevel method (this method, described below, is discussed in detail in Chapter 2). This method, which comes from multigrid algorithms used as part of some of the simulations in physics, coarsens the graph recursively to partition by matching3 edges which merge vertices and edges, then partitions the coarsest graph obtained with a global method, and gradually projects the result onto the more refined graphs [BAR 94, HEN 95, KAR 98a], as shown in Figure 4.1. During this growing phase, a refinement of the projected partition is performed each time, so that the granularity of the solution is the one of the original graphs and not that of the one of the coarsest graph.

Figure 4.1. Multilevel partitioning method. During the growing phase, at each level, the thin lines represent the projected partition obtained from the coarsest graph and the bold lines represent the refined partition

### 4.1.2.1. _Growing with refinement_

In sequential, the most popular refinement heuristics in the literature are the Kernighan-Lin KER 70] (KL) and the Fiduccia-Mattheyses [FID 82] (FM) local optimization algorithms, formulated for edges [HEN 95] or vertices [HEN 98], according to the desired separators (these methods are presented in [Chapter 2, section 2.5, page 40). All these algorithms are based on successive moves between parts of frontier vertices to reduce the current value of the cost function used. Moves that penalize the cost function can be accepted, if they are then offset by moves that provide a higher gain, allowing these algorithms to _hill-climb_ the cost function and climb out of its local minima. The vertices that have to be moved are chosen according to their _gain value_ , i.e. the value that would reduce the partitioning cost function if the vertex was moved to another part; the vertices with a negative gain value are therefore the most important ones to move. To do so, the gain values of the vertices are computed in advance, and the potentially useful vertices are inserted into appropriate data structures by using a sorting mechanism, so that the vertices with the smallest gain value can be extracted at minimal cost [HEN 95, KAR 95, KAR 98c, PEL 95, WAL 00a].

### 4.1.2.2. _Coarsening_

Coarsening and refinement are interwoven. By reducing the size of the global topological structures of the graphs into sets of several tens of vertices, the coarsening process enables the local optimization algorithms to fully consider these structures, and inversely transforms the latter into global optimization algorithms on the coarsest graphs. Then, as the uncoarsening process goes on, refinement can only focus on smaller and smaller structures of the partitions, since global structures return to their original sizes and are out-of-reach of local optimization.

However, the quality of the partitions produced by a multilevel approach may not be as good as the quality provided by global algorithms that would run for very long periods of time. Indeed, the critical point of the multilevel methods is the preservation, by coarsening algorithms, of the topological properties of the original graphs, failing which the solutions computed by initial partitioning methods would not be relevant approximations of the solutions to the original problems. Although some of the topological bias resulting from the coarsening phase can be alleviated at the finest levels by refining partitions, the local nature of the refinement phase often prevents the major modifications required to correct the most important bias which occurs. The coarsening artifacts, as much as the topology of the original graphs, can confine local optimization algorithms in the local optima of their cost function, so that the refined frontiers are often made up of non-optimal sets of locally optimal segments, as shown in Figure 4.9(a).

Most coarsening methods are based on edge matching to form the coarse vertices, so that each of these vertices contains at the most two vertices of the refined graph. Within this context, matchings are not required to be maximal; however, they must exhibit randomness properties that favor the preservation, in coarsened graphs, of the topological properties of the graphs to which they are applied.

The maximum used matching method in the literature is the " _heavy edge matching_ ", or HEM [KAR 95]. It consists of randomly selecting the vertices of the graph and matching each vertex with one of its yet unmatched neighbors that are connected by the heaviest edge, or otherwise with itself. This method facilitates the preservation of the major topological structures of graphs because, during successive matchings, the weight of uncoarsened edges increases in a way that facilitates matchings in unexplored dimensions of the mesh.

### 4.1.2.3. _Initial partitioning_

Initial partitioning algorithms aim to compute, on the coarsest graph, the partition that can be successively projected onto the finer graphs. As the quality of partitioning is imperative, expensive algorithms can be used for this purpose, especially when they operate on small graphs.

Many global methods have been suggested to compute graph partitions _ab initio_ , directly or in a multilevel context: metaheuritics (including simulated annealing [BOL 88, KIR 83], genetic algorithms [BUI 96, HOL 75], ant colonies [LAN 99], global iterative algorithms [BAT 99, LAG 94]), graph algorithms [FAH 88, KAR 95], inertial geometric methods [NOU 86], spectral methods [BAR 94], region growing algorithms [DIE 98b], etc. Although influenced by radically different fields such as genetics of statistical physics, these methods have many similarities, as well as crossed implementations, so that it is sometimes difficult to categorize them without ambiguity.

We will restrict ourselves to simply mention these different methods. Indeed, the approach we have chosen to discuss in this chapter is the parallelization of the multilevel scheme, whose initial partitioning methods remain sequential. Only the parallelization of genetic algorithms will be addressed, in another context.

Therefore, in the remainder of this chapter, after presenting the data structures required for the storage of the distributed graphs, we will describe the major parallel algorithms for coarsening and refinement in the following sections, and we will present some of the results obtained from the software PARMETIS and PT-SCOTCH.

# 4.2. Distributed data structures

The size of the graphs handled, and the need to effectively carry out parallel computations on these graphs, require a balanced distribution of the data, i.e. the adjacency information associated with each vertex, across processes.

The majority of the suggested implementations are based on distributed memory parallelism [CHE 08, KAR 98b, SCO, JOS, MET], since it is unlikely that large computers with hundreds of thousands of processing elements can offer an effective shared memory mechanism, were it NUMA4. The distributed memory parallelism paradigm has a major impact on algorithms, because the issues of mutual exclusion and lock, which are implicitly solved by memory access in sequential, must be re-formulated in terms of message exchanges, leading to latency and non-determinism, for matching as well as for refinement.

The distribution of the adjacency data, i.e. the extra-diagonal entries of the graphs adjacency matrix, can be carried out in several ways. In most of the cases, we assume that the graphs to be handled have reasonably small and bounded degrees, and that the adjacency of the vertices will always be stored locally for each process without causing any imbalance, or shortage of memory; this corresponds to a full row (or a distribution column) distribution of the entries of the adjacency matrix. To prevent imbalance when some of the rows are very dense, it is also possible to distribute by blocks the extradiagonal entries on the different processes, load balance being managed by column fragments, and not by full rows as previously [MON, VAS 05]. Because the adjacency of each vertex is distributed over several processes, the complexity of search algorithms is considerably increased, as well as some communications, because each process must store a local copy of the information associated with the vertices whose part of the adjacency belongs to the process, in order to update the known neighboring vertices.

For the smooth running of many algorithms, as well as for storing the information associated with the local vertices which it hosts, each process must maintain certain state variables of the neighboring vertices hosted by other processes. To do so, the local subgraphs of each process have local copies of the distant neighboring vertices, called _ghost vertices_ or _halo_ (see Figure 4.2). Since only the local vertices are handled by distributed algorithms, the adjacency of the ghost vertices is never stored, which guarantees the " _scalability_ " 5 of the data structures, since no process may have to store data of a size greater than the number of the edges incident to its local vertices. In order to store the information associated with these vertices in a compact manner, without using advanced data structures (hash tables or trees) [WAL 97], the local and ghost vertices are, in most of the implementations, locally reordered. This local ordering also provides an advantage of easily propagating the information of the state of the local vertices on

Figure 4.2. Data structures maintained by the software PT-SCOTCH to represent a graph distributed over three processes. The global image of the graph is shown at the top, and the three graph fragments held by the three processes are represented at the bottom. The adjacency arrays that contain the global indices of the vertices are stored in the table edgeloctab, and the compact local indices of the local and ghost vertices are stored in the table edgegsttab. The local vertices held by each process are drawn in white, and the ghost vertices in black. For each local vertex of index i located on the process p, and whose global index is given by (procvrttab[p] + _i_ \- baseval), the starting index of the adjacency table of i in edgeloctab (global numbers) or edgegsttab (local numbers) is given by vertloctab[ _i_ ], and the posterior index to the ending index by vendloctab[ _i_ ]. For example, the local vertex 2 on the process 1 has 12 for global number; its starting index in the adjacency array is 2, and its post-end index is 5; it has therefore 3 neighbors, whose global indices are 19, 2, and 11 in edgeloctab

their distant ghost copies, and grouping the ghost vertices by distant process number, which makes sending messages and/or _all-to-all_ collective communications easier.

# 4.3. Parallelization of the coarsening phase

As seen previously, multilevel methods are extremely powerful tools for graph partitioning, provided that we have unbiased coarsening and refinement algorithms for the classes of graphs considered.

We will present the different ways of carrying out the graph coarsening phase in parallel in this section, by successively approaching its different key points: matching, folding, and centralization of the coarsest graph in order to compute its initial partition.

## 4.3.1. _Construction of the coarse graph_

The parallel creation of a coarse distributed graph from a finer distributed graph is relatively simple, once each vertex has been allocated to the group which will form its future coarse vertex. By performing a halo exchange of the group numbers carried by each fine vertex, each of them would know the group numbers of each of its neighbors. The coarse adjacencies of each vertex contributing to a given group can therefore be merged on each respective process that will host them, and the adjacency of the coarse vertex will be created by eliminating the duplicate numbers of the coarse neighbors from the received lists of neighbors.

The key point of the parallel coarsening phase is therefore to devise effective and unbiased parallel matching algorithms, i.e. algorithms that can produce coarse graphs whose topological properties are independent of the distribution of refined vertices on the parallel program processes.

## 4.3.2. _Parallel matching algorithms_

Parallel matching algorithms are subjects of active research, because maximal matchings are essential for solving many problems of combinatorial scientific computing [CHA 08, HEN 06]. In the case of graph coarsening, this problem is less constrained, because matchings do not have to be maximal, but must exhibit randomness properties which allow the preservation, in coarse graphs, of the topological properties of the refined graphs to which they are applied. Otherwise, the projection of the partitions computed on coarse graphs onto the finer graphs might not represent a good solution to the initial problem, because of bias introduced by coarsening artifacts.

The fundamental problem during parallel computing of matchings on distributed graphs is the tie breaking of dependencies between matching decisions for edges incident to several processes. Any process aiming to match one of its local vertices to an adjacent distant vertex must produce a question-answer communication with the process that has the distant vertex. Because of the additional cost due to these communications and to their latency, the matching requests are usually aggregated by neighboring process, leading to an algorithm in two stages. During the first stage, the processes send the matching requests to their neighbors. During the second stage, their neighbors respond positively or negatively, depending on whether the requested vertices are free, or have already been matched previously, or are temporarily unavailable because they are also requesting another distant matching. The asynchrony between requests and responses creates collisions and dependency chains which slow the algorithm convergence. For example, when several vertices distributed over several processes request a matching with the same vertex, only one of the requests can be satisfied. Moreover, when the requested vertex has also made a request, none of the requests can be satisfied, because the vertex cannot find out if its own request would be satisfied before responding. This phenomenon is amplified with the probability that the neighboring vertices are distant: when graphs are randomly distributed (which is usually the case before they are partitioned for the first time), when their degree is high, and/or when the number of processes is large.

Several solutions to the problem of collisions between matchings have been suggested. They differ by the granularity of the mechanism of exclusion, or of reduction, of distant collisions.

## 4.3.3. _Collision reduction at process level_

The easiest way to avoid dependency chains consists of preventing communications between processes hosting fragments of the distributed graph. It then consists of favoring local matchings over distant matchings, to reduce the impact of communications on the convergence speed of the matching algorithm. In that case, as for the sequential algorithm, managing the collisions is naturally solved by the sequentiality of the computations performed within each fragment [WAL 97].

However, the effectiveness of this approach strongly depends on the initial distribution of the vertices on the processes. In the case of a random distribution, the probability of two neighboring vertices to be located on the same process decreases with the number of processes involved. That explains why, as a precaution, an iterative distant matching method can be implemented, but it causes dramatic fall in the global matching performance.

Depending on the distribution of the vertices of the graph, this matching method can lead to extremely heavy biases in coarse graphs. Therefore, its use should be restricted to cases where the bias will be as low as possible, i.e. when the number of ghost vertices (distant possible candidates for the matching) is sufficiently small compared with the number of local vertices. This can only be the case for the dynamic repartitioning of graphs that have already been distributed, or for graphs that have been distributed by low-cost geometrical partitioning, when the geometrical data are available and when the geometry of the mesh is simple enough to be efficiently separated by plans. In the case of repartitioning, local matching can be mandatory, so as to ensure that all the refined vertices of a coarse vertex belong to the same part, in order to compute the migration costs of the coarse vertices [SCH 01].

## 4.3.4. _Collision reduction at vertex level_

### 4.3.4.1. _Matching by colors_

The first solution to solve collisions at the vertex level has been suggested by Barnard [BAR 95] within the framework of the parallel formulation of a spectral partitioning multilevel algorithm [POT 90], and is based on the parallel formulation of Luby's coloring algorithm [LUB 86] to create maximal sets of independent vertices. The logic behind the use of coloring is to avoid any dependency chain, by preventing the potentially requested vertices to make a request.

During the matching algorithm, each color is considered one after another, and only the vertices of the current color can make matching requests. Since, by definition, the coloring in independent sets ensures that two neighboring vertices are not of the same color, the requested vertices cannot make requests, and therefore will always respond to their requesters. Colors are processed one after another, as many times as necessary, until all the vertices are matched or a minimum matching threshold is reached [KAR 97, KAR 99]. This algorithm does not guarantee the absence of collisions between requests, as two vertices of the same color can always send a request to a common neighbor of a different color. However, reducing the number of vertices which are able to make a request each time step, significantly reduces the number of such collisions.

The parallel formulation of Luby's algorithm works as follows. Firstly, a random number is assigned to each vertex, and a halo exchange is performed such that all the vertices know the number of each of their neighbors, whether local or distant. Then, the vertices having a number greater than the numbers of all their neighbors are marked with the first color, after which they are removed from the graph. The remaining vertices having the greatest number among their remaining neighbors are marked with the second color and are also removed, and so on until all the vertices have been handled.

This algorithm is simple and elegant, but not without flaws. The sizes of the color sets are almost always unbalanced, the last ones being often much smaller than the first ones; very often, the last set has a single vertex only.

This unbalanced distribution penalizes convergence time, to compute the coloring, as well as the matching. Because the last stages of the coloring algorithm involve

Table 4.1. Sizes of the Luby color sets for the graph 10 MILLIONS. Our version of Luby's algorithm, applied to the set of vertices of the randomly permuted graph, returns 24 sets of colors with the above distribution. Data extracted from [HER 11]

small numbers of vertices only, not much useful work is done in comparison with the additional communication cost induced, which is dominated by the network latency, leading to poor scalability. That explains why, in at least one implementation [KAR 99], the coloring algorithm is stopped prematurely, because the loss in coarsening quality resulting from a lack, in the process, of matching the fraction of the uncolored vertices has been considered as insignificant with respect to the overall gain in processing time. Regarding the matching algorithm, large sets of vertices increase the probability that multiple requests are directed toward the same vertices, therefore reducing their probability of success, whereas small sets lead to significant additional communication costs. A solution to this problem would be to alter Luby's algorithm in order to improve the balance of the color sets. However, this would often have no impact on the number of these sets, i.e. the number of communication phases.

### 4.3.4.2. _Restricted matching_

All the reasons explained above have led to the early replacement, in the tool PARMETIS, of the matching algorithm with a more direct coloring algorithm [KAR 97, section 4], which works as follows.

Initially, all the local and halo vertices are marked as unmatched. The algorithm performs a fixed number of passes (4 in the actual version), during which all the vertices which are not yet matched are considered randomly, using a random permutation. The current vertex chooses its potential mate using the HEM method. If this partner is local, both vertices are immediately marked as matched. Otherwise, the algorithm's behavior depends on the validity of a condition that involves the parity of the pass number and the global indices of the vertices requested or making a request. If this condition is met, the matching request is stored in the message intended for the corresponding process, to be sent at the end of the pass, and the vertex is marked as taken. Otherwise, nothing happens. However, in both cases, the ghost vertex that represents the distant vertex is marked as taken, so that no other local vertex can request it. At the end of the pass, the request messages are exchanged and received in an undefined order. The requests are read and agreed or rejected by the addressed processes, according to whether their local vertices have already been matched or not, and the corresponding responses are aggregated and sent back to the senders of the requests.

If the sending condition was not met, almost all the possible matchings would send requests, overwhelming the algorithm, since almost no receivers would be ready to receive, since they are also senders. By approximately preventing half the requests to be sent, and by letting their disappointed senders free to match themselves as receivers, the algorithm improves the convergence speed of the matching. It has, however, some biases, that we will discuss below.

### 4.3.4.3. _Probabilistic matching_

In order to reduce the quantity of matching messages sent at each time step, and therefore increase the probability that the receivers are able to accept these matchings, we can let a vertex send a request only if the random number that it picks is greater than a given threshold [CHE 07, CHE 08]. In the first implementations of this principle within the software PT-SCOTCH, this probability depended on the internal (local neighbors) and external (distant neighbors) degrees of the vertices, but it has been shown that a basic approach, independent of the distribution, used to work just as well, considering that a vertex was a sender with a probability of 1/2 [HER 11].

The probabilistic algorithm is made of successive matching passes, that proceed as follows. Initially, all the local vertices are randomly placed in a queue. During a pass, the vertices still in queue are considered one after another. For each of them, a random binary value is drawn. If it equals zero, the vertex is placed back in the queue for the next pass; otherwise, one of its neighbors, not yet matched, is chosen using the HEM method. If the chosen vertex is local, the matching is immediately accepted; otherwise, a matching request is stored in the message intended for the corresponding neighboring process. After all the vertices of the queue have been processed, the matching request messages are exchanged between processes, and are randomly processed by their addressees. Each request is then processed in message order. If the requested vertex is also requesting a vertex that is not the vertex which initiated the request, no response will be sent, thus indicating a collision; the matching has not been successful this time, but could be during the next pass. Otherwise, if the vertex is not yet matched, the matching is accepted. Else, a negative response will be made, so that the ghost copy of the distant vertex is marked as already matched on the process that has sent the request (however, the other neighboring processes will not be informed, until one of their local vertices sends a request). Once all the requests have been processed, the response messages are sent back, and the matching data of the sending vertices and of the ghost copies of their neighbors are modified accordingly. The above pass is repeated a defined number of times, after which a final local matching pass is performed to match locally, with a local neighbor, or with itself, any remaining vertex still in queue.

Although caused by the same logic (reducing additional communication costs and leaving enough inactive vertices to increase the probability of success of the matchings during each pass), the probabilistic algorithm of PT-SCOTCH differs from the restricted

Table 4.2. Average and average deviation of the percentage of processed vertices, and of the coarsening rate of the processed vertices, after completion of each one of the five probabilistic collective passes (C) and after the final local pass (FL). These data have been collected from the recursive coarsening of a set of test graphs on a number of processes ranging from 2 to 512. Data extracted from [HER 11]

matching algorithm implemented within the software PARMETIS. First, the decision to keep a vertex active is taken individually for each vertex, and does not depend on the choice of its potential partner and its number. Then, the probabilistic activity test concerns all the vertices, and not only the vertices located on the border of the fragments and requesting a matching with a neighbor. This avoids a bias on the restricted algorithm, where all the vertices internal to the fragments are matched during the first pass, thus favoring local matching over distant matching, because with the latter, the internal vertices are more likely to be matched with the vertices at the frontier that are next to them.

The convergence speed of the probabilistic matching algorithm can be assessed using two measures. The first one is the rate of matched vertices, computed on the number of fine vertices. This number is obvisouly equal to 100% after the final pass. The second measure is the coarsening rate among the matched vertices, i.e. the number of coarse vertices already created, taken on the number of matched fine vertices. By nature, it ranges between 50%, when all the refined vertices have been pairwise matched, and 100%, when no neighbor has been found and all the coarse vertices comprise only a single refined vertex.

As an illustration, Table 4.2 presents the data collected during the recursive coarsening, down to a thousand vertices per process, of many test graphs, for a number of processes ranging from 2 to 512. It shows the average value and the average deviation of both measures for each probabilistic collective pass, then for the final local pass6. The number of probabilistic passes has been set to 5, to maintain it small as compared to the number of passes of the coloring matching algorithm; it is only one more than the number of passes of the matching algorithm of PARMETIS. Assuming, roughly, a matching probability is slightly less than 0.5 (which depends on the probability of the requested vertex to be active and on the collision probability between matching requests, which depends on the topology of the graph), 5 collective passes have been considered as sufficient to match more than 80% of the vertices.

The experimental results confirm these assumptions, with more than 80% of matched vertices in average, and a very low coarsening rate of 53.7%. This rate shows that the distant matching is effective, and therefore the topological biases due to the initial distribution of the vertices on the processes are unlikely, as the biases induced by the final local pass, which concerns on an average only 15% of the vertices.

# 4.4. Folding

During the coarsening phase of a multilevel framework, the _folding_ is the process by which the vertices of a coarse graph are redistributed on a smaller number of processes than those on the fine graph. Performing such a folding has several advantages.

Firstly, it helps to reduce communication overhead. As the coarsening takes place, the number of vertices per process is bound to decrease, and the communication time would be dominated by set-up time and latency. Using too many processes for too little data makes collective communications sensitive to the operating system hazards, and increases the probability of increased latency. Secondly, concentrating the vertices on a smaller number of processes increases the probably of reducing the number of distant vertices per process, leading to smaller messages. Thirdly, for biased algorithms favoring local computations, increasing the number of local vertices can improve the quality of the result they produce [KAR 98d].

In most of the implementations, when a subgraph is created on a subset of processes, the remaining processes remain inactive until they are requested again during the uncoarsening phase [KAR 97]. In the meantime, they can, however, be used for computing the initial partition at the coarsest level in parallel, once the distributed coarse graph obtained is small enough to be centralized on a single process.

As part of the software PT-SCOTCH, a folding with duplication method has been implemented, to take advantage of the unused processing elements. It can be set to act in two different ways. With the first one, all the coarse vertices are stored on their local processes (i.e. the processes that have at least one of the end vertices of the coarsened edges), as represented in the first stages of Figure 4.3, which decreases the number of vertices owned by each process and accelerates future computing. With the second one, the coarse graphs are folded and duplicated, as shown in the following stages

Figure 4.3. Diagram of the parallel computation of the separator of a graph distributed on four processes, by parallel coarsening with folding-duplication, multisequential computing of initial partitions locally prolonged and refined on each process, then parallel uncoarsening of the best partition found

of Figure 4.3, which increases the number of copies of the graph and can reduce the communication (as already explained above), but also improve the final quality of the separators. Indeed, each working copy then behaves as an fully independent multilevel process, capable of providing a solution that is independent of the neighboring copies. It is therefore possible, at each level of the uncoarsening phase, to choose the best partition between the two which have been computed on both folded graphs.

The problem with the folding with duplication method is that it consumes a lot of memory. When no folding is done, and in the case of a perfect matching, the coarsening process creates, on each process, a fragment of a coarse graph whose size is half the size of the refined graph, and so on, so that the total memory taken up on each process is roughly twice the memory of the original graph7. When folding is used, each process gets two fragments of the coarse graph, one of them coming from another process, so that the size of the folded fragment is approximately equal to the size of the fragment of the refined graph. The memory footprint of the folding-duplication framework is therefore logarithmic in the number of processes8, and can consume all the available memory, if the number of processes involved increases. Therefore, as in [KAR 97], the strategy used consists of resorting to the folding only when the number of vertices of the distributed graph decreases below a minimum threshold. This threshold can define a compromise between the degree of completion of the independent multilevel executions resulting from the first stages of coarsening, which improve partitioning quality, and memory consumption.

# 4.5. Centralization

Once the distributed graph has been coarsened until it becomes smaller than a threshold size, an initial partition must be computed. Although parallel methods can be used for this purpose, such as genetic algorithms, all the implementations that we know rely on tried and tested sequential methods, because of the great quality of the partitions they produce, at a reasonable cost for such small graphs.

When simple folding is used, the coarsest graphs are already condensed on a small number of processes, and centralizing them on a single one is therefore not very expensive. Several different methods have been implemented in order to take advantage of other processes, which would be inactive otherwise. In [KAR 99], where the initial partition was computed by recursive bipartitioning, the centralized graph is duplicated on all the processes, and each of them sequentially computed the branch of the recursive bipartitioning tree leading to the part which it was in charge of. With the above approach, instead of computing the initial partition in a logarithmic time against the number of processes, each branch is independently computed in a time twice the computing time of the first bipartition9. This gain, even though real in percentage, is limited in absolute value, because it affects only small graphs. Besides, this approach can only be used when the bipartitioning tasks are totally independent of each other, which prevents its use for static mapping [PEL 94, PEL 08].

In PT-SCOTCH, when the folding-with-duplication method is used, only two processes at the most can have the same coarse graph, since both copies of a folded graph at a given time are subsequently subject to a fully independent coarsening process, using a different pseudo-random seed. Unlike the bipartition method [KAR 99], for which all the centralised graphs are similar and the same bipartition is identically computed on all the processes, the folding-duplication can compute different initial partitions for each process. This multisequential phase is shown at the bottom of Figure 4.3: the methods from the sequential library SCOTCH are used by each process to end the coarsening phase, compute the initial partition, and project it back to the largest centralized coarse graph stored on each of the processes. Then, the partitions are projected in parallel, by selecting the best out of the two available partitions for each level, when folding with duplication has been used.

The benefit of using different initial partitions for the quality of the partitions produced has been demonstrated in [CHE 07]. Experimental results integrating the use of folding-with-duplication will be presented in section 4.7.

Figure 4.4. Penalty of the cut function by the simultaneous movement of two vertices whose individual move would have improved it

# 4.6. Parallelization of the refinement phase

Because the local optimization methods used for the refinement of partitions in the sequential multilevel methods were so robust and effective, many efforts have been made in attempt to parallelize them. However, a proverb from the "parallelists" community says that the best parallel algorithm to solve a given problem is most often not the parallel transposition of the best known sequential algorithm. The refinement of the partitions is unfortunately not an exception to this rule, because the classic sequential methods possess strong sequentiality constraints, that prevent their scalable parallel direct transposition, so global methods have also been considered, the cost of which had to be reduced.

## 4.6.1. _Parallelization of the local refinement methods_

### 4.6.1.1. _Moves collisions_

As we have seen in section 4.1.2.1, local partition refinement algorithms are based on successive moves of vertices located on the frontier of the partition to reduce the current value of the cut function. In the sequential case, the vertices are moved one after another, and the gains of the chosen vertex and of its neighbors are recomputed immediately after, so that the result of a sequence of moves is always known. However, this is no longer the case if the moves can be made independently in parallel, as shown in the unfortunately famous example of the neighbors swap described in Figure 4.4.

If these two vertices had been far apart, the problem described above would not have occurred. The problem is triggered by the fact the vertices are neighbors, and that the mutual impact of their move cannot be processed in time to prevent the second move, as soon as the first one has occured. Indeed, if both vertices are located on different processes, there is no method by which a process can instantly inform its peers. This is actually not desirable, because the constraint of having to update the gain values of the distant neighboring vertices would create temporal dependencies, which would heavily penalize the scalability of the algorithm.

It follows from the previous example that, for a parallel version of a FM algorithm to work, two neighboring vertices should never be considered simultaneously. This is a very heavy constraint, which imposes a heavy burden on the programmer. As the use of dynamic distant locking mechanisms on the vertices would lead to too many communications, or even deadlocks, this constraint must be set _a priori_ , and with certainty, at the core of the algorithm. Several solutions have been suggested in the literature, which differ in the granularity of their exclusion mechanisms, in a similar way as the matching problem (see section 4.3.2). They are all based on passes during which the exclusion constraint is implemented, with a synchronization phase occurring at the end of each pass, in order to recompute the gains of the distant neighbors.

### 4.6.1.2. _Collision exclusion at vertex level_

The first solution, suggested by Karypis and Kumar, ensures neighbor exclusion at the vertex level [KAR 99]. Again, it is based on a coloring of the distributed graph, which is computed using the distributed Luby's algorithm (see section 4.3.4.1), to distribute the vertices into groups such that two neighboring vertices cannot belong to the same group. The groups associated with the colors are then considered one after another, and for each group, all the vertices of negative gain (those that improve the cut function) are moved to their favorite part. Then, the information related to the movement of these vertices is exchanged between neighboring processes, so that the gains from their neighbors are recomputed before the next group is processed. The vertices are not physically moved from a process to another, but see their part label change on the process that hosts them, as well as on the neighboring processes which keep ghost copies as distant neighbors of their local vertices.

Although this method is very scalable, due to the fact that the computations have a strong probability of being fairly distributed between all the processes (supposing that the numbers of vertices of a given color are roughly the same on each process), it loses the capacity to climb the local minima of the cost function, which was the strength of its sequential version, and therefore produces partitions of lesser quality than the latter. Indeed, in the sequential version, it was possible to accept moves of positive gain, hoping that subsequent moves of negative gain could globally reduce the value of the cost function. By uncoupling the moves of neighboring vertices in different passes, which may not occur consecutively, the algorithm prevents itself from accepting moves of positive gain. Otherwise, many processes could simultaneously decide, during the same pass, to choose vertices of positive gain, that could, moreover, not be globally smaller, without giving the algorithm the possibility of subsequently compensating the collective penalty resulting from these moves. As a result, this approach can only function as a gradient descent, without the possibility of climbing the local minima.

In fact, as the number of communication passes induced by the coloring could, again, be very high, as for the matching, this refinement method has also been discarded in the software PARMETIS, in favor of a variant of the second solution, which we will now describe.

### _4.6.1.3. _Collision exclusion at process level__

The second solution, suggested by Walshaw and Cross and implemented within the software PARJOSTLE, ensures the exclusion of neighbors at the level of interfaces between parts [WAL 00b]. Again, the algorithm functions as two nested loops. At an external level, the interface regions between the different parts are separated into disconnected zones, which are centralized on one of the two processes bordering them. These subgraphs are handled as independent problems, to which sequential local optimization methods are applied. All the local moves are then prolonged onto the whole distributed graph.

This algorithm preserves, in each interface subgraph, the capacity of the local optimization algorithm applied to it to climb out of the local minima, in this case a KL algorithm. Although its impact is limited, because the interface regions are most often long and thin, it helps to smooth the frontiers of the partition, by allowing the vertices to line up with their neighbors which have already moved along the structure of the graph.

However, what is gained in quality is lost in effectiveness. The interface subgraphs are arbitrarily assigned to the processes, which can lead to a significant load imbalance, especially when some faces would require more work than others. Moreover, with this method, the vertices are physically moved while changing part, so that each process only has the vertices of the part it represents. This can lead to an additional load imbalance, because the parts having fewer, but heavier vertices represent less work for these graph algorithms, than the parts filled with many vertices of low weight. All these factors tend to significantly penalize the scalability of this algorithm [WAL 00b, page 1653]. Besides, the connection between parts of the partition and process reduces the possibility for the algorithm to run efficiently on a number of processes which is not a divisor of the number of parts. This design choice results from the fact that PARJOSTLE has mainly been conceived as a parallel dynamic repartitioning tool.

In the method that has been implemented within the software PARMETIS [KAR 97, section 4.4], interfaces between parts are not explicitly handled. Instead, for each pass, all the moves of vertices that could improve the cut are made, as long as a condition, which depends on the pass number and a pseudo-random function based on the source and destination parts numbers, is met. This condition aims to exclude collisions at the interface level: during a given pass, for each pair of source and destination parts, the moves are accepted only in one of the two possible directions, which changes with the parity of the pass number. Therefore, swaps of neighbors cannot occur, because one of the two moves will always be rejected. Although this method is much simpler to implement than the centralization of interface zones seen previously, it can, again, only behave as a gradient descent, without the possibility of climbing the local minima.

Figure 4.5. Diagram of a banded multilevel refinement in the case of a bipartition. A small band graph is created around the prolonged frontier, whose anchor vertices represent the set of vertices removed from each part. After an optimization algorithm has been applied on the band graph, the refined frontier is prolonged onto the whole graph, after what the uncoarsening process carries on

## 4.6.2. _Band graphs_

Since local optimization parallel methods encounter many obstacles, several authors have used global methods. These methods, too expensive to be applied to large graphs in a sequential case (see section 4.1.2.3 for a list of the most popular methods), offer most often a great parallelization potential. However, due to their global nature, their cost can be high, even in parallel, and their use within a multilevel framework is, somehow, contrary to the nature of the latter, which is designed to use fast algorithms operating in reduced problem spaces. The technique described here precisely aims to reduce the space explored by global methods, in order to make them more competitive.

By the essence of the multilevel framework, it appears that, due to the local nature of KL or FM algorithms, as well as of the growing phase, it is very likely that a refined partition computed at a given level only slightly differs from the prolonged partition, since the latter is the prolongation of a partition that was a local optimum at the previous level. Indeed, to refine a partition, FM algorithms run on a small "band" of vertices surrounding the frontier of the prolonged partition.

It is therefore possible, within the uncoarsening phase of the multilevel framework, to implement a band graph extraction mechanism [CHE 06], that only considers the vertices located at a small distance of the frontiers of the prolonged partition, so that the optimization algorithms apply to this band graph rather than whole graphs, as shown in Figure 4.5 in the sequential case. The vertices that do not belong to the band graph are merged as _anchor vertices_ of weight equivalent to the sum of the weights of the merged vertices.

In [CHE 06], it has been shown that more than 98% of the vertices of the refined separators were located at a distance less than three vertices away from the projected separators, and the result of the refinements on the band graph was of an equivalent or slightly greater quality, than the quality of the refinements on the whole graph, at a slight additional cost only. This can be interpreted as the fact that the use of band graphs prevents local optimization algorithms from exploring and be trapped by local optima that would be too far from the global optimum outlined on the coarsest graph. The

Figure 4.6. Computing of a distributed band graph from a bipartitioned graph distributed on three processes. The plain line represents the current frontier of the partition, while the dotted line represents the limits of the processes between fragments of the distributed graph. The merged vertices in each part are now represented by cliques of local anchor vertices, one per process and per represented part

optimal breadth of the band graph, equal to three, is significant as such: it represents the maximum distance which can separate two refined vertices when the coarse vertices to which they belong are neighbors. Effectively, keeping more layers of vertices in the band graph is not useful, since allowing the refined separator to move away from a distance of more than 3 from the projected separator in the refined graph means that the separator could already have reached this position in the coarse graph, save for coarsening artifacts which are precisely what we want to avoid. Moreover, even if the border can only move by no more than three vertices in one level, it can move by the same amount of vertices at each following level to reach its local optimum, if needs be.

The distributed band graphs do not differ much from their sequential version presented above [CHE 08, HER 11]. Considering a distributed graph and a projected separator, also potentially distributed on several processes, the vertices closest to the separator are marked by spreading the distance informations, from neighbor to neighbor, by using halo communications; a breadth-first search of the vertices from the vertices at the border of the partition is carried out, and is limited by the breadth of the desired band graph. Then, the distributed band graph is created by adding to each process as many anchor vertices as there are parts represented in each fragment of the distributed graph.

While the sequential band graph has one anchor vertex only by part, in the parallel case, there is one anchor vertex per part and per process, as shown in Figure 4.6, in order to avoid some vertices which are of too high a degree, which would be contrary to the distributed graph model generally admitted. However, the anchor vertices of the same part form strongly coupled distributed cliques, to prevent one of the vertices from changing part in the course of optimization algorithms. In the case of a large number of processes, a hierarchical structure must be set up, again, to avoid the presence of vertices of too high a degree.

Although the distribution of the data of the band graphs can be strongly unbalanced, as some processes can be completely inactive, there is still no plan to redistribute the

Figure 4.7. Diagram of the multi-sequential refinement of a separator projected from a distributed graph on four processes

data of the band graphs before calling the optimization parallel methods, because the cost of the redistribution is considered too important with respect to the anticipated gain10. However, in the case of executions on very large parallel machines, where the collective communication costs may be very important, it might be interesting to perform a folding-with-duplication of the band graphs distributed on several subsets of processes, and to run refinement methods, independently and in parallel, on each of them.

## 4.6.3. _Multi-centralization_

The implementation of distributed band graphs in PT-SCOTCH can be associated with a multi-centralization refinement algorithm, shown in Figure 4.7. At each uncoarsening stage of the multilevel framework, a distributed band graph is created. Centralized copies of this graph are then built on each of these processes, which can then run completely independent instances of local optimization sequential algorithms such as FM. The perturbation, on each process, of the initial states of these algorithms allows each of them to explore a slightly different problem space, and therefore to improve the quality of the refinement. At the end, the best separator found is projected onto the whole distributed graph.

This method is, obviously, not scalable. There will always be a point from which centralized graphs will no longer fit in the processing elements memory, and where the execution of so many local optimization algorithms on the multi-centralized band graphs would be a waste of time, since, despite the initial perturbations, most local optima found in this way would be identical.

However, as a temporary solution, it has proved to be effective to retain the quality of the vertex separators used for the computation, on hundreds of processors, of reordering by nested dissection of graphs of a size going up to 82 millions vertices, necessary to the parallel solver PaStiX [FAV 08, HÉN 02]. The reduction of the problem space, made possible by band graphs, is significant enough to allow the computing of high quality partitions of 3D meshes of nearly a billion vertices. As the estimated size of the separators of a 3D graph of _n_ vertices is O( _n_ 2/3) [SIM 95], the order of magnitude of these separators would be around a million vertices, which can still be handled sequentially.

## 4.6.4. _Parallelization of the global refinement methods_

As we have stated previously, since the local optimization methods are difficult to parallelize, the global methods have evoked a certain interest in terms of parallelization, especially since the technique of the band graphs presented above can reduce their problem space by several orders of magnitude, as compared to whole graphs.

### _4.6.4.1. _Refinement by genetic algorithms__

The class of global methods which has been the most widely used in the literature is the class of evolutionary algorithms, and more specifically genetic algorithms [ARE 04, BUI 96, CHE 06, RAM 09, RAM 02, TAL 91]. Genetic algorithms (GA) are metaheuristics used to solve multi-criteria optimization problems by using an evolutionary method (for an introduction to these methods, refer to [WHI 94]). They consist in iteratively simulating, generation after generation, the evolution of a population of individuals that each represent a solution to the given problem, by selecting the most suitable individuals to generate the next generation. Genetic algorithms (GA) are renowned to converge very slowly, and therefore cannot be applied to large graphs [ARE 04, BUI 96].

Talbi and Bessière [TAL 91] were among the first to use parallel GA to solve the graph partitioning problem. In their implementation, the individuals that represent the solutions to the _k_ -partitioning problem are encoded as a vector, of a size equal to the number of vertices, and where each cell holds the part number to which is allocated the vertex of corresponding index. The population is evenly distributed across all the processes. In order to reduce communications and increase parallelism, all the individuals located on the same process are considered as an isolated population (also called _deme_ ) living on an island [WHI 99]. Only occasionally a few "champions" move from an island to another, to share their advantageous chromosomes with other populations trapped by local optima. The exchange of champions occurs only between neighboring processing elements, so as not to create additional communication costs.

In order to accelerate the convergence of GA, the descendants produced by crossingover and mutation can be improved, by using dedicated optimization algorithms, before integrating them to the next generation. In the case of graph partitioning, this can be done by applying a local optimization algorithm such as KL, to refine the partitions associated with each newly created individual [ARE 04, BUI 96, RAM 02]. Although this method can be interesting at first sight, since it greatly accelerates the convergence of the GA in the first generations, it can penalize its effectiveness thereafter, by considerably reducing the genetic diversity toward the norm set by the local optimization algorithm, making it extremely difficult for individuals to escape from local minima to which it brings them back.

If the use of refinement methods within a GA is not advisable, the use of a GA as refinement method within a multilevel method seems much more promising [KAV 03, RAM 02]. Both implementations quoted are sequential. In [KAV 03], the refinement method by GA is applied to the whole graph during the uncoarsening phase, leading to very significant processing times, while in [RAM 02] the GA are only applied to groups of 50 to 80 frontier vertices, in order to smooth the partition, after a greedy re-balancing method has been applied to the prolonged partition.

The second article also introduces a method called LRGA, in which the refinement by GA is performed only on a thin band of vertices around the current separator. In this GA, transformations occur with a higher probability for frontier vertices (where this behavior can be desirable to start the realignment of a whole row of vertices along the faces of a set of elements) than for internal vertices (where this behavior is not likely to bring any improvement, on the contrary). However, unlike the band graph method previously presented, this refinement method by GA on a band graph is not used within a multilevel framework, and is therefore unlikely to compute good quality partitions.

In [CHE 06] the use of a GA is presented as a parellel refinement method within a multilevel framework. Since the domain of application aimed at is sparse matrix reordering, the subjects of the population represent vertex separators11; each vertex can therefore belong to three different domains: the separator, or one of the two separated parts. A subject is therefore represented by a chromosome vector that associates a number between 0 and 2 with each vertex index. As the mutation operator can create inadequate individuals, where two vertices belonging to the two separated parts can be directly in contact, the individuals created are post-processed in order to add buffer vertices in the separator, if necessary. On the other hand, this post-processing does not attempt to optimize the quality of the individuals, so as not to disrupt the smooth running of the GA. The results presented, based on a multi-deme algorithm, show that genetic algorithms can indeed replace, in terms of quality, the KL or FM local optimization methods. Regarding processing times, by reducing the work space of the GA to the band graph surrounding the current frontier, the processing times measured in sequential (execution on a single process) are only 30 to 80 times more than the processing time of an optimized FM algorithm, for a yielded quality equivalent to the latter. Due to the almost perfect scalability of the GA, their implementation in parallel as refinement algorithms, working on distributed band graphs, is very promising, when computing time is not a critical factor.

### _4.6.4.2. _Refinement by diffusion methods__

Many authors have noted that the partitions computed by local optimization algorithms are not optimal. One of the most critical communities is the community of users of iterative methods to solve linear systems [DIE 98b, VAN 95], for whom these partitions are not always suitable, because parts with extended frontiers and irregular shapes make it necessary to use a greater number of iterations for the resolution to converge.

To measure the quality of each of these parts, several authors go by a measure called _aspect ratio_ , which can be defined, in _d_ dimensions, as the ratio between the size of the interface of a part and the   power of its content12 [DIE 98a, DIE 98b, RAM 09]. The more a part is compact, the more its _aspect ratio_ value is small.

In [DIE 98b], Diekmann _et al_. have highlighted such a behavior, and have suggested a measure of the _aspect ratio_ of the parts and a set of heuristics to create and refine the partitions, with the aim to reduce this _aspect ratio_. Among these algorithms, there is a bubble growing algorithm, which is physically analogous to the growing of soap bubbles that, by coming in contact with one another, seek to minimize the surface of their interfaces. A disadvantage of these methods is that they do not ensure that all the parts would be of identical size, which requires the subsequent use of load balancing algorithms [DIE 98b, MEY 05]. In [MEY 05], Meyerhenke and Schamberger explored this model further, and suggested a way to make the bubbles bigger by solving, eventually in parallel, a system of equations, instead of iteratively computing the centers of the bubbles. This method gives partitions of very high quality, but is very slow, even in parallel [MEY 06] and despite the use of band graphs [MEY 08], especially since the load balance problem remains.

As mentioned above, the GA have very interesting characteristics, from a scalability and quality point of view. However, their convergence rate is low, because they are not specialized enough toward the problem to be solved. In order to save

Figure 4.8. Diagram of the diffusion model of the jug of the Danaides

computing power and time, it is necessary to have scalable global methods, which are specifically conceived to give short and smooth frontiers. The _jug of the Danaides_ algorithm [PEL 07] has been devised for this purpose.

This algorithm can be applied to any number of parts, but for simplicity, its description will be based on the bipartitioning case. The graph to be partitioned is modeled as shown in Figure 4.8. The vertices are represented as barrels of infinite capacity, which can leak, so that at most one unit of liquid can escape per unit of time. When the graphs are weighted, the maximum quantity of liquid that can be lost per unit of time is equal to the weight of the vertex. The edges of the graph are modeled by pipes of section equal to their weight. In each part, a source vertex receives   units of liquid per unit of time. In the case of bipartitioning, two types of liquid are injected in the system: scotch in the first pipe, and anti-scotch in the second one, so that, when a certain quantity of liquid is in contact with the same quantity of its antagonist, both disappear. To simplify the writing of the algorithm in the bipartitioning case, the scotch is represented by positive quantities, and the anti-scotch by negative quantities.

The diffusion works as summarized in Algorithm 4.1. At each time step, and for each vertex, the quantity of liquid (scotch or anti-scotch) that remains in each barrel after part of it has leaked is distributed through the pipes, according to their respective sections, toward the neighboring barrels. This process could be carried out until convergence, but is actually run only a limited number of times, until it reaches the stability of the signs of the values contained in the barrels. Indeed, what matters is not the final quantities of liquid contained in each barrel, but to know which barrels contain scotch or anti-scotch, i.e. where the resulting frontier is.

As | _V_ | units of both liquids are injected on the whole per unit of time, and as the barrels can let the same quantity leak in the same time, the system is bound to converge, especially since some liquid also disappears from the collision between scotch and anti-scotch. As for the bubble-growing methods, the result expected is the creation of a smooth front between parts. The aim of this algorithm is more, for this

* * *

**Algorithm 4.1.** Outline of the diffusion algorithm of the _jug of the Danaïdes_. The scotch, represented by positive quantities, flows from the source of part 1, while the anti-scotch, represented by negative quantities, flows from the source of part 0. At each time step, the current content of the barrels and the new content are respectively stored in the tables old and new

* * *

* * *

purpose, to get a global smoothing of the frontier than a strict minimization of the cut. Indeed, unlike diffusion algorithms presented previously, this method favors the load balance over the cut minimization.

As for bubble-growing algorithms, the principle of the _Danaides jug_ problem is the choice of the source vertices. The anchor vertices of the band graphs represent a natural choice: since they are connected to all the vertices of the last layer of vertices, the liquids flow as a front, as if they were coming from a source far apart from the frontier, which is analogous to what would occur if they were coming from the center of a bubble having the frontier as its perimeter.

The frontiers obtained from this method are of good quality [PEL 07], as shown in Figure 4.9(b), and the parts are balanced if the number of iterations is high enough. Additional optimizations, on the number of iterations and the multiplication of the quantities of liquid within the barrels, can make the sign of the barrels converge in around forty iterations only, and give, on one processor, results similar to those given by an optimized FM algorithm, in only three times longer.

Figure 4.9. Partition of the graph altr4 into 8 parts using Scotch 5.0.6, with three different strategies. The frontiers segmented by the FM algorithm are clearly visible in (a). The pure diffusion method returns the smoothest frontiers, as shown in (b). The combination of the two gives the result in (c). These images are in color on the Website of this book, perso.ec-lyon.fr/charlesedmond.bichot/livre_partitionnement

The advantage of this algorithm is that it is highly scalable. Its parallel version only slightly differs from its sequential version [HER 11]. It is based on the data propagation methods between local and ghost vertices to spread the quantities of liquid to diffuse, per unit of edge weight, to the neighboring vertices.

# 4.7. Experimental results

The results obtained, in the case of graph partitioning, with the most recent versions of the parallel tools PT-SCOTCH (version 5.1.6) and PARMETIS (version 3.1) are presented below. This version of PT-SCOTCH includes all its methods described in the previous sections. During those tests, PT-SCOTCH has been used to compute graph partitions by recursive bipartition, using probabilistic matching, folding-withduplication at the lowest levels of coarsening, and diffusion on distributed band graphs. PARMETIS has been used in a common way, in direct _k_ -partitioning.

Our experiments were carried out on the supercomputer Platine of the CCRT13. This machine is a Bull Novascale cluster made of 932 computing nodes interconnected by an Infiniband network. Each node comprises four dual-core Intel Itanium II processors. The measure used to estimate the quality of the partitions is the cut, where the parts supposed are to be balanced.

Table 4.3 presents the processing times and the cut obtained with PT-SCOTCH and PARMETIS, for a significant set of test graphs (these graphs are presented in [HER 11]), for representative numbers of processes and parts.

In most of the test cases, the partitions computed by PT-SCOTCH are of quality equivalent or better than the quality produced by PARMETIS, with a gain that can reach nearly 20% during bipartitioning of the graph 82 millions, regardless of the number of processes used. PT-SCOTCH always computes better results for small amounts of parts, while PARMETIS produces cuts marginally better for three graphs, which are AUDIKW1, THREAD and BRGM, when the number of parts increases. This phenomena is due to the fact that, currently, PT-SCOTCH performs its _k_ -partitionings by using recursive bipartitionings, while PARMETIS uses a direct _k_ -partitioning algorithm. In fact, the quality of the partitions produced by PT-SCOTCH is likely to deteriorate for great numbers of parts, because of the greedy nature of the recursive bipartitioning method, which prevents us from reconsidering the bipartitioning choices already made. It is therefore not surprising that the three graphs for which PARMETIS surpasses PTScotch are those of greater average degree, for which the poor decisions made at previous stages produce greater penalties in terms of cost.

The negative impact of recursive bipartitioning is even more visible regarding processing time. While PT-SCOTCH can be more than three times faster than PARMETIS in the case of a simple bipartitioning, as for example for the graph 82MILLIONS, when the number of parts increases, its processing time shows a penalty that tends to a constant proportional to the inverse of the coarsening rate of the graphs.

However, the loss of quality due to bipartitioning is most often compensated by the improvement of the quality brought by folding-duplication and multi-sequential phases. Actually, for most of the graphs, PT-SCOTCH as much as PARMETIS offer a stable quality of partition for any number of parts and up to 384 processes (which is the greater number considered in these experiments). However, for certain graphs, such as cage15, the quality of partitioning of PARMETIS deteriorates as the number of processes increases, while it increases for PT-SCOTCH. This graph, which is not a mesh, has many topological irregularities (in terms of degree and connectivity) that are likely to create coarsening artifacts and, therefore, require an effective local optimization during the growing phase. As stated in section 4.6.1.2, and already demonstrated in the case of graph reordering [CHE 08], the difficulty of the FM implementation of PARMETIS to climb the local minima of the cut function restricts its effectiveness, when the number of processes increases.

In order to analyze the scalability in time of PT-SCOTCH, we can focus on simple bipartitioning. For all the graphs studied, PT-SCOTCH remains perfectly scalable until 64 processors, then its scalability decreases, more quickly for smallest graphs. It remains satisfactory until 256 processors for the largest graph, 82MILLIONS. As already stated in [KAR 99], a significant increase in the size of the graphs is necessary to retain a parallel effectiveness constant for PARMETIS, and PT-SCOTCH is no exception either.

As a final comparison between the two tools, the last column of the Table 4.3 presents the best processing time, and the associated cut, obtained by PT-SCOTCH and PARMETIS, during the simple bipartitioning of each of the graphs. We can see that PT-SCOTCH produces most often partitions of better quality in a shorter time. In addition, PT-SCOTCH reaches its peak performance for a smaller amount of processes

Table 4.3. Comparison between PT-SCOTCH (PTS) and PARMETIS (PM) for numbers that represent processes and parts. CPTS and CPM , and tPTS and tPM, are respectively the sizes of the edge cuts, and the processing time in seconds, for PTS and PM. Ppeak is the number of processors on which PTS and PM have recorded the best time for each graph. The numbers between brackets, just after the processing times, show the value of Ppeak. Data extracted from [HER 11]

Table 4.4. Continuation of Table 4.3. Daggers indicate an ending due to an invalid MPI operation

than PARMETIS for mesh graphs. This lack of stability is, in our opinion, due to the use of multi-centralization, which is absolutely not scalable and leads to many data moves, when the number of processes increases.

# 4.8. Conclusion

At present, there are already several effective tools [JOS, MET, SCO] to partition effectively in parallel many families of graphs (in particular mesh graphs). However, the scalability of these tools for large numbers of processes still poses problems, and continues to be the subject of intensive research.

# 4.9. Bibliography

[ARE 04] AREIBI S., ZENG Y., "Effective memetic algorithms for VLSI design automation = genetic algorithms + local search + multilevel clustering", _Evolutionary Computation_ , vol. 12, no. 3, p. 327-353, 2004.

[BAR 94] BARNARD S.T., SIMON H.D., "A fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems", _Concurrency: Practice and Experience_ , vol. 6, no. 2, p. 101-117, 1994.

[BAR 95] BARNARD S.T., "PMRSB: Parallel multilevel recursive spectral bisection", _Proceedings of ACM/IEEE Conference on Supercomputing (CDROM)_ , December 1995.

[BAT 99] BATTITI R., BERTOSSI A.A., "Greedy, prohibition, and reactive heuristics for graph partitioning", _IEEE Transactions on Computers_ , vol. 48, no. 4, p. 361-385, 1999.

[BOL 88] BOLLINGER S.W., MIDKIFF S.F., "Processor and link assignment in multicomputers using simulated annealing", _Proceedings 11th International Conference on Parallel Processing_ , The Penn State University Press, p. 1-7, August 1988.

[BUI 96] BUI T.N., MOON B.R., "Genetic algorithm and graph partitioning", _IEEE Transactions on Computers_ , vol. 45, no. 7, p. 841-855, 1996.

[CHA 08] CHAN A., DEHNE F., BOSE P., LATZEL M., "Coarse grained parallel algorithms for graph matching", _Parallel Computing_ , vol. 34, no. 1, p. 47-62, 2008.

[CHE 06] CHEVALIER C., PELLEGRINI F., "Improvement of the efficiency of genetic algorithms for scalable parallel graph partitioning in a multilevel framework", _Proceeding of European Conference on Parallel Computing (Euro-Par'o6)_ , Dresden, vol. 4128, Lecture Notes in Computer Science, p. 243-252, September 2006.

[CHE 07] CHEVALIER C., Conception et mise en oeuvre d'outils efficaces pour le partitionnement et la distribution parallèles de problèmes numériques de très grande taille, PhD thesis, LaBRI, University of Bordeaux I, September 2007.

[CHE 08] CHEVALIER C., PELLEGRINI F., "PT-SCOTCH: a tool for efficient parallel graph ordering", _Parallel Computing_ , vol. 34, p. 318-331, 2008.

[DIE 98a] DIEKMANN R., MEYER D., MONIEN B., "Parallel decomposition of unstructured FEM-meshes", _Concurrency: Practice & Experience_, vol. 10, no. 1, p. 53-72, 1998.

[DIE 98b] DIEKMANN R., PREIS R., SCHLIMBACH F., WALSHAW C., "Aspect ratio for mesh partitioning", _Proceedings of European Conference on Parallel Computing (Euro-Par'98)_ , vol. 1470, Lecture Notes in Computer Science, p. 347-351, 1998.

[FAH 88] FAHRAT C., "A simple and efficient automatic FEM domain decomposer", _Computers and Structures_ , vol. 28, no. 5, p. 579-602, 1988.

[FAV 08] FAVERGE M., LACOSTE X., RAMET P., "A NUMA aware scheduler for a parallel sparse direct solver", _Proceedings of Parallel Matrix Algorithms and Applications_ , June 2008.

[FID 82] FIDUCCIA C.M., MATTHEYSES R.M., "A linear-time heuristic for improving network partitions", _Proceedings of 19th Design Automation Conference_ , IEEE, p. 175-181, 1982.

[HEN 95] HENDRICKSON B., LELAND R., "A multilevel algorithm for partitioning graphs", _Proceedings of ACM/IEEE Conference on Supercomputing (CDROM)_ , p. 28-es, December 1995.

[HEN 98] HENDRICKSON B., ROTHBERG E., "Improving the runtime and quality of nested dissection ordering", _SIAM Journal of Scientific Computing_ , vol. 20, no. 2, p. 468-489, 1998.

[HEN 06] HENDRICKSON B., POTHEN A., "Combinatorial scientific computing: the enabling power of discrete algorithms in computational science", _Proceedings of the Vector and Parallel Processing Conference_ , p. 260-280, 2006.

[HÉN 02] HÉNON P., RAMET P., ROMAN J., "PASTIX: A high-performance parallel direct solver for sparse symmetric definite systems", _Parallel Computing_ , vol. 28, no. 2, p. 301-321, January 2002.

HER 11] HER J.H., PELLEGRINI F., "Efficient and scalable parallel graph partitioning", _Parallel Computing_ , 2009, Submitted. [www.labri.fr/~pelegrin/papers/scotch_parallelbipart_parcomp.pdf.

[HOL 75] Holland J., _Adaptation in Natural and Artificial Systems_ , University of Michigan Press, Ann Harbor, 1975.

JOS] "JOSTLE: Graph partitioning software", [staffweb.cms.gre.ac.uk/~c.walshaw/jostle/.

[KAR 95] KARYPIS G., KUMAR V., "Multilevel graph partitioning schemes", _Proceedings of the 24th International Conference on Parallel Processing, III_ , CRC Press, p. 113-122, 1995.

[KAR 97] KARYPIS G., KUMAR V., "A coarse-grain parallel formulation of multilevel k-way graph-partitioning algorithm", _Proceedings of 8th SIAM Conference on Parallel Processing for Scientific Computing_ , 1997.

[KAR 98a] KARYPIS G., KUMAR V., "A fast and high quality multilevel scheme for partitioning irregular graphs", SIAM Journal on Scientific Computing, vol. 20, no. 1, p. 359-392, 1998.

[KAR 98b] KARYPIS G., KUMAR V., METIS, A Software Package for Partitioning Unstructured Graphs, Partitioning Meshes, and Computing Fill-Reducing Orderings of Sparse Matrices, University of Minnesota, September 1998.

[KAR 98c] KARYPIS G., KUMAR V., "Multilevel _k_ -way partitioning scheme for irregular graphs", _Journal of Parallel and Distributed Computing_ , vol. 48, p. 96-129, 1998.

[KAR 98d] KARYPIS G., KUMAR V., "A parallel algorithm for multilevel graph partitioning and sparse matrix ordering", _Journal of Parallel and Distributed Computing_ , vol. 48, p. 71-95, 1998.

[KAR 99] KARYPIS G., KUMAR V., "Parallel multilevel _k_ -way partitioning scheme for irregular graphs", _SIAM Review_ , vol. 41, no. 2, p. 278-300, 1999.

[KAV 03] KAVEH A., RAHIMI H.A.B., "A hybrid graph-genetic method for domain decomposition", _Finite Elements in Analysis and Design_ , vol. 29, p. 1237-1247, 2003.

[KER 70] KERNIGHAN B.W., LIN S., "An efficient heuristic procedure for partitionning graphs", _BELL System Technical Journal_ , p. 291-307, February 1970.

[KIR 83] KIRKPATRICK S., GELATT C.D., VECCHI M.P., "Optimization by simulated annealing", _Science_ , vol. 220, no. 4598, p. 671-680, May 1983.

[LAG 94] LAGUNA M., FEO T.A., ELROD H.C., "A greedy randomized adaptative search procedure for the two-partition problem", _Operations Research_ , vol. 42, p. 677-687, August 1994.

[LAN 99] LANGHAM A.E., GRANT P.W., "Using competing ant colonies to solve k-way partitioning problems with foraging and raiding strategies", ECAL '99: _Proceedings of the 5th European Conference on Advances in Artificial Life_ , p. 621-625, 1999.

[LUB 86] LUBY M., "A simple parallel algorithm for the maximal independent set problem", _SIAM Journal of Computing_ , vol. 15, no. 4, p. 1036-1055, 1986.

MET] METIS: Family of Multilevel Partitioning Algorithms, [glaros.dtc.umn.edu/gkhome/views/metis.

[MEY 05] MEYERHENKE H., SCHAMBERGER S., "Balancing parallel adaptive FEM computations by solving systems of linear equations", _Proceedings of European Conference on Parallel Computing_ (Euro-Par '05), vol. 3648 of Lecture Notes in Computer Science, p. 209-219, 2005.

[MEY 06] MEYERHENKE H., SCHAMBERGER S., "A parallel shape optimizing load balancer", _Proceedings of European Conference on Parallel Computing (Euro-Par '06)_ , vol. 4128 of Lecture Notes in Computer Science, p. 232-242, 2006.

[MEY 08] MEYERHENKE H., MONIEN B., SAUERWALD T., "A new diffusion-based multilevel algorithm for computing graph partitions of very high quality", _Proceedings of 22nd Parallel and Distributed Processing Symposium_ , p. 1-13, 2008.

MON] MONDRIAAN for sparse matrix partitioning, [www.math.uu.nl/people/bisseling/Mondriaan/.

[NOU 86] NOUR-OMID B., RAEFSKY A., LYZENGA G., "Solving finite element equations on concurrent computers", Noor A.K. (ed.), _Parallel Computations and their Impact on Mechanics_ , ASME Press, p. 209-227, 1986.

[PEL 94] PELLEGRINI F., "Static mapping by dual recursive bipartitioning of process and architecture graphs", _Proceedings of Scalable High Performance Computing Conference (SHPCC '94)_ , IEEE, p. 486-493, May 1994.

[PEL 95] PELLEGRINI F., Application de méthodes de partition à la résolution de problèmes de graphes issus du parallélisme, PhD thesis, LaBRI, University of Bordeaux I, January 1995.

[PEL 07] PELLEGRINI F., "A parallelisable multilevel banded diffusion scheme for computing balanced partitions with smooth boundaries", _Proceedings of European Conference on Parallel Computing (Euro-Par '07)_ , vol. 4641 of Lecture Notes in Computer Science, Springer, p. 191-200, August 2007.

PEL 08] PELLEGRINI F., SCOTCH and LIBSCOTCH 5.1 User's Guide, LaBRI, University of Bordeaux I, August 2008, [www.labri.fr/~pelegrin/scotch/.

[POT 90] POTHEN A., SIMON H.D., LIOU K.P., "Partitioning sparse matrices with eigenvectors of graphs", _SIAM Journal of Matrix Analysis_ , vol. 11, no. 3, p. 430-452, July 1990.

[RAM 02] RAMA MOHAN RAO A., APPA RAO T.V.S.R., DATTAGURU B., "Automatic decomposition of unstructured meshes employing genetic algorithms for parallel FEM computations", _International Journal of Structural Engineering and Mechanics_ , vol. 14, p. 625-647, 2002.

[RAM 09] RAMA MOHAN RAO A., "Distributed evolutionary multiobjective mesh-partitioning algorithm for parallel finite element computations", _Computers & Structures_, vol. 87, no. 23- 24, p. 1461-1473, 2009.

[SCH 01] SCHLOEGEL K., KARYPIS G., KUMAR V., "Wavefront diffusion and LMSR: algorithms for dynamic repartitioning of adaptive meshes", _IEEE Transactions on Parallel and Distributed Systems_ , vol. 12, no. 5, p. 451-466, 2001.

SCO] Scotch: Static mapping, graph partitioning, and sparse matrix block ordering package, [www.labri.fr/~pelegrin/scotch/.

[SIM 95] SIMON H.D., TENG S.H., "How good is recursive bipartition", _SIAM Journal of Scientific Computing_ , vol. 18, no. 5, p. 1436-1445, 1995.

[TAL 91] TALBI E.G., BESSIÈRE P., "A parallel genetic algorithm for the graph partitioning problem", _ICS'91: Proceedings of the 5th International Conference on Supercomputing_ , ACM, p. 312-320, 1991.

[VAN 95] VANDERSTRAETEN R., KEUNINGS R., FARHAT C., "Beyond conventional mesh partitioning algorithms", _SIAM Conference on Parallel Processing_ , p. 611-614, 1995.

[VAS 05] VASTENHOUW B., BISSELING R.H., "A two-dimensional data distribution method for parallel sparse matrix-vector multiplication", _SIAM Review_ , vol. 47, no. 1, p. 67-95, 2005.

[WAL 97] WALSHAW C., CROSS M., EVERETT M.G., "Parallel dynamic graph partitioning for adaptive unstructured meshes", _Journal of Parallel Distributed Computing_ , vol. 47, no. 2, p. 102-108, 1997.

[WAL 00a] WALSHAW C., CROSS M., "Mesh partitioning: a multilevel balancing and refinement algorithm", _SIAM Journal of Scientific Computing_ , vol. 22, no. 1, p. 63-80, 2000.

[WAL 00b] WALSHAW C., CROSS M., "Parallel optimization algorithms for multilevel mesh partitioning", _Parallel Computing_ , vol. 26, no. 12, p. 1635-1660, 2000.

[WHI 94] WHITLEY D., "A genetic algorithm tutorial", _Mathematics and Statistics_ , vol. 4, p. 65-85, June 1994.

[WHI 99] WHITLEY D., RANA S., HECKENDORN R.B., "The island model genetic algorithm: on separability, population size and convergence", _Journal of Computing and Information Technology_ , vol. 7, p. 33-47, 1999.

1 Chapter written by François PELLEGRINI.

1 Depending on the equipment's characteristics, we can have several processing units or elements per processor at our disposal, and/or several processors. The _processing element_ term includes all these possibilities.

2 These graphs have good locality and parting properties in common: their maximum degree is small, and their diameter evolves at least as a fractional power of their size, for example,   for _d_ -dimensional meshes of size _n_.

3 Graph matching is defined in Chapter 2, page 34.

4 " _Non-uniform memory access_ ".

5 This term describes the ability of an algorithm to require a number of resources (memory and/or time) per processing element which is inversely proportional to the number of elements used to run it. Ideally, running an algorithm on twice the number of processing elements would reduce the time and/or memory per element by half, and then this algorithm is said to be perfectly " _scalable_ ". In practice, in most cases, the overhead induced by parallelism generates a loss of performance that is expressed, in the function defining the consumption of resources according to the number of elements, as a multiplicative entry greater than 1, often logarithmic. We will then commonly refer to " _weak scalability_ ".

6 Such a local pass will produce the same kind of biases as the criticized ones from the restricted matching algorithm of PARMETIS. However, unlike the latter, these biases are not inherent to the algorithm, but depend on the number of probabilistic passes that the user is ready to perform beforehand.

7 Let _T_ ( _x_ ) be the size of a fragment of a graph representing a fraction _x_ of the initial number of local vertices, linear in _x_ because the degree is supposed to be limited. The total storage cost per process is therefore:

8 The storage cost is equal to:

9 Computations are similar to those mentioned in the previous note.

10 Especially since, in the current implementation of the band graphs within the software PT-SCOTCH for problems of medium size, the distributed band graphs are centralized just after they have been created, which will be explained further.

11 However, the solutions implemented can also apply to edge separators, in a similar way to the one presented in [TAL 91].

12 For example, for a 2D mesh, the _aspect ratio_ of a part is measured as the ratio between the length of a perimeter of the part, against the square root of its surface. The smallest value of this metric is reached for parts of circular shape in the 2D Euclidean space.

13 The Research and Technology Computing Center, www-ccrt.cea.fr/.

# Chapter 5

# Static Mapping of Process Graphs 1

This chapter presents different state-of-the-art techniques used to solve an extension to the general graph partitioning problem, called "static mapping", which is commonly used to assign the communicating processes of a parallel program to a nonhomogeneous parallel architecture.

# 5.1. Introduction

The development of multicore processors and large parallel machines, exceeding a thousand processing elements, no longer allows their users to benefit from a uniform view on memory and network. These heterogeneous architectures are collectively designed under the name of "NUMA" architectures (for "non-uniform memory architecture"), even in the case of distributed memory machines for which the data exchange is performed by means of a heterogeneous interconnection network. The hierarchical structure of these architectures, in particular, regarding the memory access (existence of multiple levels of cache and communication buses, shared by an increasing number of processing elements as we move away from them), has a very strong impact on the performance of the programs running on it. The more the data to be loaded from remote locations, the higher the latency of the transfer will be, due to the increasing distance, and the lesser the bandwith will be, due to the sharing of the bandwith of the communication channel by a greater number of entities1.

At the level of each execution flow, it is therefore essential to maximize the locality of memory accesses, in order to benefit the most from cache effects. At the level of the parallel application, it is important to favor the locality of the exchanges between execution flows, whether these exchanges are carried out through shared memory or an interconnection network.

The effective execution of a parallel program on a parallel machine therefore requires us to minimize the time spent to achieve the communications induced by parallelization, while balancing through time the computing load assigned to each processor, in order to completely utilize the parallelism.

This operation is called _scheduling_ if we consider the logical and temporal dependencies of processes, and _mapping_ if we suppose that all the processes simultaneously coexist on the parallel machine during the entire duration of the execution of the program.

A mapping is said to be _static_ if it is computed prior to the execution of the program and is never reconsidered, and _dynamic_ if the processes can be moved during the execution.

The scheduling problem has been the subject of many contributions, mainly from the community of real-time systems, for which these questions are critical [ORO 97]. The most commonly used model consists of representing the application as a directed acyclic graph (or DAG), on which a critical path search is run [ESH 97, HAR 99, KHA 94].

The static mapping problem has rather been dealt with by the community of parallel and distributed programming. This problem is NP-hard in its general form, as well as in many restricted forms [GAR 79]; only in very specific cases that this problem can be solved using polynomial-time algorithms (see section 5.3).

# 5.2. Static mapping models

The computation of an effective static mapping requires an _a priori_ knowledge of the dynamic behavior of the target machine toward the program that is run. To do so, we should be able to model the interactions between the program and the machine, by isolating the decisive criteria of the quality of a mapping, to extract from the parallel program the information required for its mapping according to the defined criteria, and to find the mapping of the program optimizing these criteria.

## 5.2.1. _Cost functions_

Modeling the quality of a mapping is usually carried out using cost functions, which determine from their content the characteristics of the optimal mapping that we wish to obtain. The main criteria quoted in the literature [AND 88, MUN 91a, PLA 91, TAL 92, UÇA 06, and their references], and that can take place in these functions are: the total time of use of the machine, i.e. the duration of the execution of the longest process, which conditions the termination of the parallel program; the load balance of the computation across the different processors: the processing time of the program is directly linked to it since, if communication is not taken into account, an equidistributed load ensures a minimal processing time; the load balance of incoming and outgoing communication of the processors; the load balance of the communication within processors, i.e. the communication between processes mapped onto the same processors; the load balance of the communication through the physical links of the machine; the minimization of the total communication cost, as the sum of the costs of the communication channels of the parallel program weighted by a function depending on the mapping. This function takes into account the shortest path distance between the processors where the end processes of the channel are assigned.

In most of the cases, the cost functions combine several of these criteria by summing and weighting them with parameters that eventually depend on the characteristics of the target machine. Thus, if a machine computes quickly but communicates slowly, we would put a higher penalty on communication than on computation, in order to favor mappings that minimize communication.

The major disadvantage of the unified cost functions is the existence of weighting coefficients. Besides the fact that these coefficients must be measured for each target machine, the mixture of quantities of a different nature, even if they are handled as numbers without dimension (differences relative to the average, for example), is in practice not easy to perform.

Some authors therefore prefer to separate computation from communication. That is, a communication cost function is minimized, while keeping the load balance within a predefined tolerance [ERC 90, PEL 94, WAL 95]. This could be translated, within a single cost function, by the fact of adding to the communication function a term of zero value if the balance is met, and infinite in the opposite case, in order to select only the solutions enforcing the load balance. However, this formulation strongly compromises some mapping algorithms, such as genetic algorithms [ARE 04, MUN 91b] or simulated annealing [BOL 88, KIR 83] that, while being able to handle discontinuous cost functions, take advantage of the slight energy threshold between neighboring states to explore the problem space arround the current solution. In the case of simulated annealing, a new configuration which improves the cost function is always accepted (we thus naturally converge to local minima), while a worse configuration is accepted with a probability that is all the more low as the energy difference with the previous configuration is high. The fact of adding such an "all or nothing" penalty term therefore creates insurmountable energy "walls", which strongly constrain the exploration of the solution space, thereby reducing the capacity of the algorithm to find the global minima by hill-climbing zones of higher cost, which was its strength. When significantly constrained, the algorithm spends most of its time generating unacceptable configurations. The hierarchy of the computing and communication functions mainly appears only in graph algorithms [FID 82, HEN 96, KER 70, PEL 94, WAL 95], which operate only on acceptable configurations, and therefore converge much faster for the same quality of solution2.

The cost functions to minimize belong to two main categories [ERC 90, LO 84, SAD 87a]. The first one is the category of " _minimax_ " type functions, thus called because the cost function to minimize is expressed as the maximum, on the set of target vertices, of the load assigned to each of them. In the case of process mapping, this cost function estimates the completion time of the work of each processor, depending on their computing and communication load. The second category is the category of " _global sum_ " type functions, because the quantity to minimize is expressed as the sum, on all the processors, of the communication costs and load imbalances resulting from these mappings.

_Minimax_ type functions are assumed to model more accurately the phenomenon to optimize, but are much more costly to handle, in particular, when compared to local optimization algorithms, because each modification of the current solution requires the complete recomputing of the maximum, after modification of the values of the cost function for each of the target vertices affected by the modification. On the contrary, _global sum_ type functions accept incremental modifications, but according to some authors [LO 84], they do not effectively promote load balance, since a mapping where only one target vertex would be much more heavily loaded than all the others can have the same value as a mapping where half the target vertices have a greater load than the remaining half, while the completion time associated with the first one would be much higher.

In practice, _global sum_ type functions are often used within a dissociated cost function, where a transfer is accepted only if none of the loads of each target vertex deviates from the ideal value of more than a certain percentage [ERC 90, PEL 94, HEN 96]. Therefore, even if these functions lead to more significant load imbalances than the _minimax_ functions, these imbalances are limited and do not increase with the number of target vertices, which would be the case otherwise.

The most used cost function of _global sum_ type communication is the sum, for all the edges of the source graph _S_ , of their dilation3 multiplied by their weight:

[5.1]

where _τ S,T_ is an application from the vertex set _V_ ( _S_ ) of the source graph _S_ to the vertex set _V_ ( _T_ ) of the target graph4 _T_ , returning the target vertex onto which is mapped each of the source vertices ( _τ S,T_( _v S_) = _v T_ if the source vertex _v S_ is mapped on the target vertex _v T_), and where _ρ S,T_ is an application that associates each edge { ,  } of E(S) with a chain of shortest path linking _τ S,T_ (  ) to _τ S,T_ (  ).

Irrespective of the type of routing used for the communications, this cost function models the traffic on the interconnection network of the target architecture, and then the congestion5 risk, and its minimization favors the mapping on nearby processing elements of strongly intercommunicating processes [HAM 92, PEL 94, WAL 95].

In [WAL 95], Walshaw _et al_. use, to measure the dilation of the edges on a 2D grid, not the Manhattan distance on this grid, but the square of this distance, to compromise heavily on distant communications compared to local communications. This metric has also been used in [HUA 06].

The choice of simple cost functions is justified _a posteriori_ by the fact that there is no exact correspondence between the variations of the value of a mapping and the real processing time of the parallel program mapped in this way, due to many unquantifiable hardware artifacts (hierarchy of memory accesses, scheduling of the processes by the operating system on each processing element, latency and congestion of the communications on the interconnection network, etc.) [HAM 92, SAD 87b, WAL 95]. There is, however, a strong positive correlation, which justifies the need for static mapping, when compared to simple graph partitioning.

## 5.2.2. _Heterogeneity of target architectures_

The static mapping problem has several variations, depending on the degree of heterogeneity of the target architecture considered. The higher this degree, the more complex the methods implemented to solve the problem, because of the increasing number of variables and constraints to consider.

The first type of heterogeneity concerns architectures in which the processing elements are heterogeneous but the network is homogeneous (non-valuated complete graph). This problem is therefore seen as a variation of the graph partitioning problem, in which the parts to find must be of variable sizes, corresponding to the relative weights of the different processing elements. The target architecture can therefore be represented as a vector whose weights determine the processing capacity of each of these elements [KAR 98].

The second type concerns architectures that are heterogeneous in terms of power of processing elements and communication links, but for which the execution cost of a process is the same, regardless of the processing element on which it is located. This is the problem that has so far received the most attention from the community of parallel applications developers [BOK 81a, ERC 90, HAM 92, HEN 96, HUA 06, PEL 96b, WAL 95], because it can easily model hierarchical interconnection networks of many parallel machines, workstations clusters, and computing grids.

The preferred representation of target architectures is therefore a valuated graph, commonly called "task interaction graph" (TIG) [STO 77], where the vertices represent the processing elements and edges the direct communication links between these elements6. We associate valuations with the vertices and edges of this graph, most often integers, respectively estimating the computing power of the processing elements and crossing cost (comparable to the inverse of the bandwith) of the links. These representations allow to easily sum the processing power available on a set of processing elements, and to evaluate, by summing the links values, the transfer cost of a message along a path. The implicit communication model is then a model of exchange in constant time, expressing the latency, and in which the length of the messages is not considered explicitly.

In the case of a "wormhole" communication network, in which the header-block packet of the message holds and sets a virtual path that will be used by the following data packets7, the sum of the communication costs is the sum of the latencies of each link, before the head of message reaches its destination. In the case of a "store and forward" network, in which the messages are completely stored on each intermediate node before being forwarded, the cost of the links models the total transfer cost of a message, which can be estimated with respect to small messages (for which the latency prevails) or large messages (for which the term _Lτ_ prevails).

The third and last type models the mapping problem in the most general way possible, by taking into account the heterogeneity of the computations on the processing elements: the duration of each task of the parallel program may differ, depending on the processing element on which it is located [KAY 09, UÇA 06]. This type of mapping has been increasingly used with the emergence of _Cell BE_ heterogeneous processors [ARE 08], which are made of processing elements with very different capabilities: the SPE ("synergistic processing elements") of this processor are very effective for computing operations on full small data blocks, thanks to their vector computation units, whereas the PPE ("powerpc processing element") is more suited for the execution of fragments of code requiring access to the main memory.

The affinity between the processes and the processing elements is represented as an "estimated time to compute" (or ETC) matrix providing, for each process number and each processing element, the execution cost of the process on the element considered. This matrix is accompanied by a TIG that, as for the previous model, represents the communication costs between source vertices.

# 5.3. Exact algorithms

Exact algorithms have the characteristic of always finding the solution or solutions of minimal cost, when they run successfully. Because of the exhaustive nature of their search, in a general case, they are mainly based on algorithms that systematically explore a search tree whose nodes are subsets of solutions, the path through the branches from the root to the leaves therefore representing the fact of binding the variables of the problem one after another until complete solutions are obtained. As these methods are out of the scope of this work, we will only describe them briefly, and in the following section we will rather focus on studying the methods based on graph decomposition.

The "branch and bound" methods avoid running through all the branches, by "pruning" the subtrees thereby leading to solutions worse than a given solution. To do so, we associate with each vertex a reduction, as precise as possible, of the cost function computed on the variables of the subproblem corresponding to this vertex8; by construction, this reduction is such that the value of a reduced vertex is always greater than that of the original vertex. The result is that if the value of a vertex is greater than the cost of a solution already found, there is no point in searching the corresponding subtree, as its leaves will not be better solutions. The algorithm therefore consists of, starting from an initial solution and from the root, searching the tree, by updating the current solution as soon as a solution of lower cost is found, and avoiding the branches leading to vertices of value greater than the cost of the current solution. Kafil and Ahmad [KAF 98] use this algorithm to map TIGs on heterogeneous processing elements connected by a homogeneous network, the function to minimize being the total execution time, i.e. the maximum time required by each processing element to complete its computations and communications. Kaya and Uçar later suggested a formulation suitable for heterogeneous networks [KAY 09]. Although the complexity of search algorithms is, in the worst case, similar to that of an explicit enumeration, they are, on average, more effective.

Another approach, that we will only mention, is dynamic programming. The problem is then represented as the minimization under constraints of a quadratic cost function in {0, 1} variables [BOK 81b, HAN 86, ROU 89].

It should be noted that, in some cases, the mapping problem can be reduced to a polynomial problem.

In [BOK 81b], Bokhari uses a dynamic programming algorithm to map exactly, in a time quadratic to the number of target vertices, source graphs with a tree structure.

Stone [STO 77] has also shown how an optimal mapping can be obtained in polynomial time for a biprocessor machine. This result was partially extended to any number of processing elements by Lo [LO 84], who suggests a polynomial algorithm that can lead to an optimal proven solution, but that may also fail. She also suggests an optimal polynomial algorithm in the case of identical processing elements that can only support at most two processes each [LO 88].

Stone modeled the mapping to achieve under the form of a single graph, whose vertex set contains processes as well as both processing elements. An edge between two processes is weighted by the cost of the data that is exchanged, and an edge between a process and a processing element is valued by the cost of the mapping of the process on the other element. By taking both processing elements as source and as infinite capacity well, it is then possible, using a maximal flow algorithm, to compute in polynomial time a minimal cost cut where each part contains a single processing element. This cut corresponds to the optimal mapping intended, which is performed by allocating to each element the processes located on the same part.

To extend this method, Lo used a graph similar to the one described above, but with more than two vertices representing processing elements [LO 84]. To determine the processes located on each element, we must apply the maximal flow algorithm to the graph, by taking the element considered as source, with the infinite capacity well being built by fusion into a unique vertex of all the other processing elements. The processes that have only been selected by a unique element are allocated to it, and are removed from the graph. We then apply the method again on the subgraph obtained. When the subgraph no longer contains any process, the computed mapping is optimal; if an iteration ends without any process being mapped, the algorithm fails.

The second algorithm suggested by Lo is totally different from the previous one, because it is based on the use of a maximal perfect matching [LO 88]. It is valid only if the number of processes is less than or equal to twice the number of processing elements, and if each of them accepts at the most two processes. The graph considered here is the communication graph between processes, to which we apply a polynomial algorithm to compute a maximal perfect matching. The ends of the edges obtained represent the most inter-communicating processes, and are therefore placed on the same unit. The remaining isolated vertices are then placed in pairs on the unused units.

# 5.4. Approximation algorithms

Since the entire problem is NP-hard, the production of mappings of graphs of arbitrary topoligies and of large sizes can only be considered using heuristics, in order to avoid the combinatorial explosion of the computing time.

As for the constrained graph partitioning problem, many classes of heuristics have been used, the most widely used being metaheuristics [ERC 90, HER 09, MUN 91a, MUN 91b, RAM 88, TAL 92], and graph algorithms [SAD 87a, SAD 87b, BER 87, ERC 90, MUN 91a, HAM 92, PEL 94, HEN 96, HER 09]. Based on the theme of this work, we will focus more particularly on those implementing, in an underlying or explicit way, graph partitioning methods.

## 5.4.1. _Global methods_

Global methods aim to solve the mapping problem by considering the source and target graphs in their entirety, and based on the complete cost function (unlike recursive methods, that will be addressed at a later point).

Due to the complexity of the problem to be solved, which does not allow any greedy algorithm to provide an effective solution, these algorithms are all iterative. They proceed by refinement of an existing solution, by perturbing it and keeping the best solution found during their exploration process of the solution space.

Initial solutions used as starting points by iterative algorithms are obtained using greedy partitioning methods of low cost, as for example, the Bokhari [BOK 81a] algorithm, which makes regions grow from randomly chosen seed vertices, or the Gibbs-Poole-Stockmeyer algorithm [GIB 76] that partitions a graph in bands by breadth-first search from a pseudo-peripheral vertex9 [SAD 87a]. These algorithms aim to provide an initial solution consisting of balanced and predominantly compact and

Figure 5.1. Hypercube graphs of dimensions 2 and 3, with their vertices indices written, expressed in binary

connected partitions, where the iterative algorithms seek to improve the cost function, while maintaining the balance.

### 5.4.1.1. _Direct methods_

Direct methods are global methods which are working only on the whole source and target graphs. The majority of these algorithms date from around 20 years, where source graphs (representing processes graphs in most applications) were small enough so that local optimization algorithms could provide efficient results, without resorting to more elaborated frameworks, such as multilevel algorithms which will be introduced later.

A number of iterative refinement algorithms have been developed for target graphs of a specific topology. It is for example, the case of the "cyclic pairwise exchange" algorithm of Hammond [HAM 92], which considers in turn the dimensions of the target hypercube and swaps vertices between neighboring target vertices, along the considered dimension, i.e. the binary representation, writing of the indices only differs from the bit corresponding to this dimension (see Figure 5.1).

In [WAL 95], Walshaw _et al_. consider the mapping on grids of 1- and 2D. Their initial partitioning method relies on the geometric coordinates of source graph vertices, to distribute them spatially on a straight line or on the plane; the methods used for this partition are, however, very basic, and cannot guarantee either the density or the initial balance of the parts. The refinement method that they implement seeks to reinstate the latter by shifting the most "off-centered" vertices toward the neighboring parts, thereby improving the balance and/or the communication cost function. The measure of the off-centering of a frontier vertex, in contact with at least one vertex belonging to one of the four neighboring parts (north, south, east, and west), is defined as the maximum distance between this vertex and the vertices next to the opposite parts ("north" for "south", "east" for "west", etc.)10. In order to maintain the parts in rectangular shapes, with square corners, the off-centering measure of a vertex is not computed in norm 1 (Manhattan distance, which would give diamond-shaped areas) or Euclidean (with the shape of a disc), but by using an infinite norm.

The mapping on 2D grid has also been studied by Sadayappan and Ercal, who suggest an initial mapping algorithm based on the division of the source graph into bands, and then blocks, by bands intersection, and optimizing the potentially imbalanced partition obtained by iterative migration of vertices between neighboring parts on the grid [SAD 87b].

In most of the cases presented above, the iterative migration of the vertices occurs by using a modified version of the Kernighan-Lin algorithm (this algorithm and several of its variants are presented in Chapter 2), and by more elaborated cost functions than the simple edge cut metric [SAD 87a, SAD 87b, WAL 95].

Non-specific direct methods are either numerical methods such as linear programming and quadratic assignment [HAN 86, HUA 06], or metaheuristics commonly used within graph partitioning, such as genetic algorithms [MUN 91b, TAL 92, UÇA 06] or simulated annealing [BOL 88, RAM 88, TAL 92]. As seen previously, for these methods to work effectively, it is essential that the cost function used, does not induce impassable discontinuities.

These methods are very expensive, and not suitable for large source graphs for process mapping applications. Their cost remains prohibitive even in the case of a massively parallel implementation, as it has been done for a static mapping refinement genetic algorithm, although proceeding on reduced problem spaces (band graphs [CHE 06]) in a multilevel context [HER 09] (see below).

### 5.4.1.2. _Clustering methods_

Clustering methods aim to reduce the complexity of the mapping problem, by decoupling the source graph partitioning problem and the problem of mapping the obtained parts on the target graph. Firstly, the source graph to map is divided into as many parts as the number of vertices of the target graph. Secondly, the valuated quotient graph11 resulting from the partition is mapped onto the target graph in order to minimize the communication cost function.

The clustering of vertices of the source graph can be done in two ways: either downward, by partitioning the source graph in as many part, as there are vertices in the target graph (this will be most often done by recursive bipartitioning [SAD 87a]), or upward, by hierarchical aggregation of vertices connected by edges of heaviest weight [BER 87]. In the last case, we must avoid the attraction effects to the clusters already created, which would otherwise reduce the quality of the coarse graphs, which requires to take into account other criteria than the simple reduction of communications, such as the similarity of the processing time of the clusters to merge [UÇA 06]. The mapping of the quotient graph onto the target graph is then realized using direct methods discussed previously.

### 5.4.1.3. _Multilevel methods_

Multilevel methods transpose to the static mapping problem the algorithms initially devised for graph partitioning (see Chapter 2, which is specifically dedicated to them). This transposition is simple, as coarsening and uncoarsening algorithms are identical. The only points to reconsider concern the computing of initial mappings and local optimization methods.

Multilevel methods can be seen as an optimization of the clustering methods, as the projection onto the source graph, without refinement, of the mapping computed on the coarsest graph, is equivalent to the computing of a mapping on a quotient graph obtained by clustering. The uncoarsening process of the multilevel algorithm can only bring further improvement over clustering methods.

Initial mappings, on graphs of very small size, can be computed using the direct algorithms discussed previously [ARA 08, HUA 06, UÇA 06], or using recursive methods that will be introduced in the next section [HER 09].

The mapping refinement at each level is traditionally performed by using modified versions [ARA 08, HUA 06] of the Kernighan-Lin or Fiduccia-Mattheyses algorithms [FID 82].

## 5.4.2. _Recursive methods_

As their name suggests, recursive methods are based on a series of consecutive divisions of the global problem into partial subproblems, which are easier to solve individually, to produce by successive stages the desired mapping. Unlike clustering methods which, in a first phase, cluster vertices of the source graph, before considering the mapping of the quotient graph obtained on the target graph, recursive methods take into account the topology of the target graph to compute the successive partitions of the source graph. Instead of successively and independently considering the source graph and then the target graph, these methods consider simultaneously both graphs, by gradually binding the variables of the problem to reach a solution, by progressive restriction of the subset of target vertices onto which each vertex of the source graph will be mapped. They are not, as such, direct methods, because the cost function estimated each time is not the original cost function, but a partial function meant to produce the same effect of reduction of distant communications at the final mapping level.

### 5.4.2.1. _General structure_

The first method of this kind has been suggested by Ercal _et al_. in the case of homogeneous hypercube architectures [ERC 90]. On a hypercube of dimension _d_ , whose vertices are numbered from 0 to 2 _d_ − 1, the result of the mapping algorithm is a vector providing, for each vertex of the source graph, the number of the vertex of the hypercube onto which it is mapped, written in binary, and whose bits have all been specified.

At the beginning of the algorithm, the binary values of the target vertex numbers are not known. The first bipartition of the source graph sets the first bits of each value of the solution vector to 0 or 1, depending on whether the source vertices are contained in one or the other part. From the following stage, when a bipartition of a subgraph of the source graph needs to be computed, the cost function takes into account, for each source vertex, not only the number of edges connecting it to vertices belonging to the other part (local cut of the current bipartition), but also the Hamming distance12 between the bits already known of the number of the target vertex and the vertices connected by edges already cut in the previous stages. The above information allows us to adjust the position of the frontier to the previous bipartitions, and to position the following bits according to the underlying hypercube topology, in order to minimize the number of edges of dilation greater than 1, as shown in Figure 5.2(d).

The fact of considering, within a bipartitioning algorithm, the mapping information resulting from the previous bipartitions, had already been implemented in an adaptation of the Kernighan-Lin algorithm to VLSI circuits mapping [DUN 85]. This adaptation consists of incorporating, within the cost functions of these algorithms, terms taking into account the edges of the cocycle13 of the subgraphs considered.

The principle of the Ercal _et al_. algorithm has been extended to any type of architecture, even heterogeneous, under the name of _dual recursive bipartitioning_ (DRB) [PEL 94]. This was done by generalizing the concepts of bipartition of the target architecture graph (which appeared implicitly in the successive setting of the values of the bits describing the number of the target vertex on which the vertices of

Figure 5.2. Impact of the consideration of the result of partial mappings already computed during the mapping of a graph on a square (hypercube of dimension 2). During the first bipartition (a), no information is available; during the second one (b), the mapping information relative to the neighboring part is not significant. During the third bipartition, if we do not consider the information already available (c), and if we perform a plain partitioning, a random choice of the second bit can produce a large number of edges of dilation 2 (shown on the diagram). If we consider the results from the previous bipartition, we can align the new frontier with the previous one and choose effectively the parts labeling, thus producing (d) a minimal number of edges of dilation 2

the source graph are mapped) and of distance between subparts of the same graph (the Hamming distance being no longer suitable in most cases).

The DRB algorithm proceeds by recursive allocation of source subgraphs to subsets of target vertices, until these subsets of target vertices are reduced to a single element, or until the source subgraphs are empty. At each stage, the algorithm performs the bipartition of a target subgraph, also called _domain_ , in two disconnected subdomains, and calls a graph bipartition algorithm to map the source subgraph on the two subdomains created, as described in Algorithm 5.1.

The association of a subdomain with each source vertex defines a _partial mapping_ of the source graph. As the bipartitionings take place, the size of the subdomains onto which the source vertices are mapped decreases, until a complete mapping is achieved, when all the subdomains are of size 1.

The DRB algorithm relies on four main objects: (i) a _domain_ structure, which represents a subgraph of the target graph; (ii) a _domain bipartitioning method_ that, given a domain, bipartitions it in two disconnected subdomains; (iii) a source subgraph _partial mapping method_ that, given a domain, its two subdomains, and a source subgraph, bipartitions the latter in two disconnected subsets to be mapped on the two subdomains; (iv) a _distance function_ between domains, that generates, in the target graph, an estimated measure of the distance between two domains. This function must

* * *

**Algorithm 5.1**. Outline of the mapping algorithm by dual recursive bipartitioning (DRB). At each stage, the current set of target vertices is bipartitioned, then the source subgraph is mapped on the two subdomains created, in proportion to their respective weights if they are not equal

* * *

* * *

satisfy certain homogeneity properties, such as producing more accurate results as the sizes of the domains decrease. This funtion is used, during source graphs bipartitiong, to compute, in the communication function to minimize, the dilation of the edges of the source graph. This leads to considering, in costs computing, that the routings in the target graph are of shortest paths. This assumption is not unrealistic, since the communication systems of the parallel machines dynamically manage the routing by shortest paths algorithms. The DRB algorithm therefore does not explicitly compute the routings, leaving their management to the communication system of the target machine.

All these objects are shown as black boxes by the DRB algorithm, which can then accept any type of target architecture and processes bipartitioning method.

For the final mapping to be of good quality, it is essential that each partial mapping computed during the successive bipartitions of the source subgraphs on the subdomains satisfies the globally defined criteria. The bipartition of a subgraph _S_ ′ of _S_ must therefore maintain the balance of the load of both subgraphs within the tolerance specified by the user, and the communication cost function to minimize during each partial mapping is defined as:

[5.2]

Figure 5.3. Edges considered by the partial communication cost function during the bipartition of a subgraph between the two subdomains D0 and D1 of a domain D. The dotted edges correspond to a zero dilation, both their ends being located on the same subdomain. The cost of the edges of the cocycle is determined with respect to the distance between the subdomain to which the internal end of the edge belongs, and the domain supposed to host the external end

which considers not only the dilatation of the internal edges of the subgraph, but also of the edges of the cocycle of _V_ ( _S_ ′) in _S_ , as show in Figure 5.3. This helps avoid making local choices that could be globally detrimental. This cost function has also been used in [HEN 96], who has independently implemented the DRB algorithm, by using a spectral method to compute constrained bipartitions of the source subgraphs.

When a multilevel framework is used to compute bipartitions of subgraphs, it must consider the partial mapping data already available on the neighboring vertices of the subgraph considered. This information is intended for initial bimapping computing algorithms, as well as for those designed to refine the partial bimapping prolonged during the uncoarsening phase. The latter are traditionally versions of the Kernighan- Lin or Fiduccia-Mattheyses algorithms [FID 82], able to consider these additional criteria during the computation of the gains of each vertex movement.

The affinity data of a vertex with either of both parts can be integrated in two ways. The first one consists of adding to the movement gain of each vertex an _external gain_ , representing the cost variation due to the edges of the cocycle having this vertex as an internal end. This gain is therefore equal to the sum, for each one of these edges, of the distance between the domain of the external end and the subdomain on which the internal vertex can be moved, minus the distance between the external domain and the current subdomain of the vertex, multiplied by the weight of the edge. When the vertex changes part, the sign of the external gain changes [PEL 94]. The second solution consists of adding to the graph two immutable "anchor" vertices, permanently allocated to each of the two parts. Each of the vertices of the graph having a particular affinity with one of the two parts is then connected to the anchor vertex representing it by an edge of weight equal to the additional communication cost induced by the shift of the vertex out of its main part [DUN 85, HEN 96]. Thus, local optimization algorithms do not have to be modified: moving a vertex out of its main part naturally induces an increase of the cost function, equal to the value of the penalty assigned to the edge connecting the vertex with its anchor vertex.

### 5.4.2.2. _Bipartitioning sequencing_

From the double recursive call of the algorithm follows a binary tree recursion pattern, where each node is a partial mapping task, i.e. the contrained bipartitioning of both the source and target subgraphs. Every way to traverse this tree corresponds to a sequencing of the partial mapping tasks, which directly influences the mappings obtained as each task is based on the results of the previous tasks concerning the neighboring vertices of the current subgraph. The two types of sequencing considered here are depth-first sequencing and breadth-first sequencing (i.e. by level).

In the case of a depth-first sequencing, as programmed in the above draft, the partial mapping tasks involved in the left branches of the tree have no precise data on the distances to the vertices located on the right branches (see Figure 5.4(a)). However, the tasks of the right branches have the exact mapping of the vertices located on the left branches, and thus we can estimate the cost function to minimize more accurately.

In contrast, sequencing the tasks by traversing the tree by level provides that, at each level, a constrained bipartition task can find out on which subdomains all the source vertices have been allocated at the previous level. Therefore, to decide in which subdomain to map a given source vertex, the bipartitioning tasks can consider in a better way the communication costs relative to the neighboring vertices, for which a globally more accurate distance can be computed. This results in an interesting feedback effect: once an edge has been caught in a cut between two subdomains, the distance between its ends will be considered in the partial communication cost function to minimize, and the following tasks will therefore tend to maintain these vertices close to each other, as shown in Figure 5.4(b). Moreover, as all the domains are bipartitioned at each level, they are all of equivalent size, which satisfies the homogeneity of the distance function and achieves coherency of the algorithm.

The implementation of the DRB algorithm within the software SCOTCH14 equally uses both sequencings, allowing their comparison. The partial mapping tasks are stored in a list of tasks, as executing a task causes the creation of two new tasks at the most. They are then added to the list, either to the head in the case of a depth-first search (we then have a stack structure), or to the tail in the case of a search by level (we then have a first-in first-out structure).

Figure 5.4. Consequences of the depth-first and by level sequencing on the bipartition of a domain D belonging to the leftmost branch of the bipartitioning tree. With sequencing by level, the mapping data regarding the vertices located on the right branches of the bipartitioning tree is more accurate (N.C. refers to the cut level)

However, processing tasks by level is not sufficient to ensure good neighborhood properties of the parts computed. Indeed, if the tasks are processed in a random order within one level, the parts produced by the first task can be allocated to the target vertices in a globally inconsistent manner, heavily penalizing the following tasks, as shown in Figure 5.5. The DRB algorithm implemented within the software Scotch therefore orders the tasks to process according to several criteria, such as the size of the part (the largest parts, potentially the most penalizing in case of a problem, are processed first) and/or the number of neighboring tasks already processed [PEL 08], in order to joint across neighboring domains the frontiers of the newly computed parts, as shown in Figure 5.6.

Figure 5.5. Impact of the processing order of the partial mapping tasks of last level during a mapping of a mesh on a 2×4 grid. With the processing order shown in Figure (a), if the parts computed by the tasks 1 and 3 are not mapped in the same direction, the last task will have to produce two "twisted" parts, in order to minimize the partial cost function

Figure 5.6. Mapping by level and step by step across neighboring parts of the graph of Figure 5.5. The frontiers are jointed to minimize the number of edges of dilation 2

It was shown that the search of the tree by level and by neighbors provides on average the best solutions [PEL 96a], because this collective refinement of the partial mappings tends to prevent making irremediable decisions in the left branches of the tasks tree.

From an algorithmic point of view, the recursive methods are greedy, as the results of the successive bipartitioning are never challenged. However, iterative algorithms can be used to compute them, as well as to refine the complete mapping computed by recursive methods, for example in the case of a multilevel framework, as previously discussed.

The DRB algorithm results in poor parallelization. Indeed, within a parallel context, the tasks of a same level must be processed in parallel, which leads to a loss of performance due to sequencing problems of the tasks described previously. The implementation in parallel of a static mapping computing algorithm can only be performed using global methods, preferably multilevel ones.

# 5.5. Conclusion

The static mapping problem, extensively studied in the mid-80s with the outcome in laboratories of the first parallel machines with heterogeneous memory access and interconnection networks, became obsolete while the architectural advances allowed to hide this heterogeneity to the programmer. This problem arises anew, with a renewed intensity due to the emergence of the heavily NUMA architectures, based on very heterogeneous processing element hierarchies, as well as on very large interconnection networks, most often hierarchical (parallel machines with hundreds of thousands of processing elements and computing grids).

At the same time, the size of the graphs to map has considerably increased, with the meshes handled by processes mapping exceeding a billion vertices. The mapping computation must be processed in parallel, preventing the use of the DRB algorithm, which was highly effective in sequential. Because direct methods are too expensive, the parallel multilevel methods are currently explored.

# 5.6. Bibliography

[AND 88] ANDRÉ F., PAZAT J.L., "Le placement de tâches sur des architectures parallèles", _Technique et Science Informatiques_ , p. 385-401, April 1988.

[ARA 08] ARAFEH B., DAY K., TOUZENE A., "A multilevel partitioning approach for efficient tasks allocation in heterogeneous distributed systems", _Journal of Systems Architecture_ , vol. 54, no. 5, p. 530-548, 2008.

[ARE 04] AREIBI S., ZENG Y., "Effective memetic algorithms for VLSI design automation = genetic algorithms + local search + multilevel clustering", _Evolutionary Computation_ , vol. 12, no. 3, p. 327-353, 2004.

[ARE 08] AREVALO A., MATINATA R.M., PANDIAN M., PERI E., RUBY K., THOMAS F., ALMOND C., _Programming the Cell Broadband Engine TM Architecture–Examples and Best Practices_, Redbooks, IBM, 2008.

[BER 87] BERMAN F., SNYDER L., "On mapping parallel algorithms into parallel architectures", _Journal of Parallel and Distributed Computing_ , vol. 4, no. 5, p. 439-458, 1987.

[BOK 81a] BOKHARI S.H., "On the mapping problem", _IEEE Transactions on Computers_ , vol. C-30, no. 3, p. 207-214, 1981.

[BOK 81b] BOKHARI S.H., "A shortest tree algorithm for optimal assignments across space and time in distributed processor system", _IEEE Transactions on Software Engineering_ , vol. 7, p. 583-589, 1981.

[BOL 88] BOLLINGER S.W., MIDKIFF S.F., "Processor and link assignment in multicomputers using simulated annealing", _Proceedings of 11th International Conference on Parallel Processing, Penn State University Press_ , p. 1-7, August 1988.

[CHE 06] CHEVALIER C., PELLEGRINI F., "Improvement of the efficiency of genetic algorithms for scalable parallel graph partitioning in a multilevel framework", _Proceedings of the European Conference on Parallel Computing (Euro-Par '06), Dresden_ , vol. 4128 of Lecture Notes in Computer Science, p. 243-252, September 2006.

[DUN 85] DUNLOP A.E., KERNIGHAN B.W., "A procedure for placement of standard-cell VLSI circuits", _IEEE Transactions on CAD_ , vol. CAD-4, p. 92-98, 1985.

[ERC 90] ERCAL F., RAMANUJAM J., SADAYAPPAN P., "Task allocation onto a hypercube by recursive mincut bipartitioning", _Journal of Parallel and Distributed Computing_ , vol. 10, p. 35-44, 1990.

[ESH 97] ESHAGHIAN M.M., WU Y.C., "Mapping heterogeneous task graphs onto heterogeneous system graphs", _Proceedings of Heterogeneous Computing Workshop (HCW '97)_ , p. 147-160, April 1997.

[FID 82] FIDUCCIA C.M., MATTHEYSES R.M., "A linear-time heuristic for improving network partitions", _Proceedings of the 19th Design Automation Conference_ , IEEE, p. 175-181, 1982.

[GAR 79] GAREY M.R., JOHNSON D.S., _Computers and Intractablility: A Guide to the Theory of NP-Completeness_ , W.H. Freeman, San Francisco, 1979.

[GIB 76] GIBBS N.E., POOLE W.G., STOCKMEYER P.K., "A comparison of several bandwidth and profile reduction algorithms", _ACM Transactions on Mathematical Software_ , vol. 2, p. 322-330, 1976.

[HAM 92] HAMMOND S.W., Mapping unstructured grid computations to massively parallel computers, PhD thesis, Rensselaer Polytechnic Institute, Troy, New York, February 1992.

[HAN 86] HANSEN P., GIAUQUE W.C., "Task allocation in distributed processing systems", _Operations Research Letters_ , vol. 5, no. 3, p. 137-143, August 1986.

[HAR 99] HARY S.L., ÖZGÜNER F., "Precedence-constrained task allocation onto point-to-point networks for pipelined execution", _IEEE Transactions on Parallel and Distributed Systems_ , vol. 10, no. 8, p. 838-851, 1999.

[HEN 96] HENDRICKSON B., LELAND R., VAN DRIESSCHE R., "Enhancing data locality by using terminal propagation", _Proceedings of Hawaii International Conference on System Sciences_ , IEEE, p. 565-574, January 1996.

[HER 09] HER J.H., PELLEGRINI F., "Towards efficient and scalable parallel static mapping", _CSC 2009_ , SIAM, October 2009.

[HUA 06] HUANG S., AUBANEL E., BHAVSAR V.C., "PaGrid: a mesh partitioner for computational grids", _Journal of Grid Computing_ , vol. 4, p. 71-88, 2006.

[KAF 98] KAFIL M., AHMAD I., "Optimal task assignment in heterogeneous distributed computing systems", _IEEE Concurrency_ , vol. 6, p. 42-51, 1998.

[KAR 98] KARYPIS G., KUMAR V., METIS, A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices, University of Minnesota, September 1998.

[KAY 09] KAYA K., UÇAR B., "Exact algorithms for a task assignment problem", _Parallel Processing Letters_ , vol. 19, p. 451-465, 2009.

[KER 70] KERNIGHAN B.W., LIN S., "An efficient heuristic procedure for partitioning graphs", _BELL System Technical Journal_ , vol. 49, no. 2, p. 291-307, February 1970.

[KHA 94] KHAN A.A., MCCREARY C.L., JONES M.S., "A comparison of multiprocessor scheduling heuristics", _Proceedings of International Conference on Parallel Processing_ , vol. 2, p. 243-250, 1994.

[KIR 83] KIRKPATRICK S., GELATT C.D., VECCHI M.P., "Optimization by simulated annealing", _Science_ , vol. 220, no. 4598, p. 671-680, May 1983.

[LO 84] LO V.M., "Heuristic algorithms for task assignment in distributed systems", _International Conference on Distributed Computer Systems_ , IEEE, p. 30-39, 1984.

[LO 88] LO V.M., "Algorithms for static task assignment and symmetric contraction in distributed computing systems", _Proceedings of International Conference on Parallel Processing_ , IEEE, p. 239-244, 1988.

[MUN 91a] MUNTEAN T., TALBI E.-G., "Méthodes de placement statique des processus sur architectures parallèles", _T.S.I., Technique et Science Informatiques_ , 1991.

[MUN 91b] MUNTEAN T., TALBI E.-G., "A parallel genetic algorithm for process-processors mapping", _High Performance Computing_ , vol. 2, p. 71-82, 1991.

[ORO 97] OROZCO J., CAYSSIALS R., "Precedence constraints in hard real time distributed systems", _Proceedings of 3rd IEEE International Conference on Engineering of Complex Computer Systems_ , p. 33-45, September 1997.

[PEL 94] PELLEGRINI F., "Static mapping by dual recursive bipartitioning of process and architecture graphs", _Proceedings of Scalable High-Performance Computing Conference (SHPCC-94)_ , IEEE, p. 486-493, May 1994.

[PEL 96a] PELLEGRINI F., ROMAN J., Experimental analysis of the dual recursive bipartitioning algorithm for static mapping, Research Report, LaBRI, University of Bordeaux I, August 1996.

[PEL 96b] PELLEGRINI F., ROMAN J., "SCOTCH: a software package for static mapping by dual recursive bipartitioning of process and architecture graphs", _Proceedings of High Performance Computing Networking (HPCN '96)_ , Brussels, vol. 1067 LNCS, p. 493-498, April 1996.

PEL 08] PELLEGRINI F., SCOTCH and LIBSCOTCH 5.1 User's Guide, LaBRI, University of Bordeaux I, August 2008, [www.labri.fr/~pelegrin/scotch/.

[PLA 91] PLATEAU B., RAYNAL P., TRYSTRAM D., VINCENT F., Placement de tâches: Un tour d'horizon des techniques efficaces, Report, IMAG, June 1991.

[RAM 88] RAMANUJAM J., ERCAL F., SADAYAPPAN P., "Task allocation by simulated annealing", _Proceedings of International Conference on Supercomputing_ , vol. 3, p. 471-480, 1988.

[ROU 89] ROUCAIROL C., HANSEN P., "Cut cost minimization in graph partitioning", in Brezinski, C. (ed.), _Numerical and Applied Mathematics_ , J.C. Baltzer AG., p. 585-587, 1989.

[SAD 87a] SADAYAPPAN P., ERCAL F., "Cluster partitioning approaches to mapping parallel programs onto a hypercube", _Proceedings of International Conference on Supercomputing_ , vol. 297 LNCS, p. 475-497, June 1987.

[SAD 87b] SADAYAPPAN P., ERCAL F., "Nearest-neighbor mapping of finite element graphs onto processor meshes", _IEEE Transactions on Computers_ , vol. C-36, no. 12, p. 1408-1424, December 1987.

[STO 77] STONE H.S., "Multiprocessor scheduling with the aid of network flow algorithms", _IEEE Transactions on Software Engineering_ , vol. SE-3, no. 1, p. 85-93, January 1977.

[TAL 92] TALBI E.G., MUNTEAN T., Évaluation et étude comparative d'algorithmes d'optimisation combinatoire : application au problème de placement de processus, Report no. RR 886-I, LGI-IMAG, April 1992.

[UÇA 06] Uçar B., AYKANAT C., KAYA K., IKINCI M., "Task assignment in heterogeneous computing systems", _Journal of Parallel and Distributed Computing_ , vol. 66, no. 1, p. 32- 46, 2006.

[WAL 95] WALSHAW C., CROSS M., EVERETT M.G., JOHNSON S., MCMANUS K., "Partitioning and mapping of unstructured meshes to parallel machine topologies", _Proceedings of Irregular '95_ , vol. 980 of Lecture Notes in Computer Science, p. 121-126, 1995.

1 Chapter written by François PELLEGRINI.

1 The latency of a transfer is the duration between the start of the request and the reception of the first data requested. The bandwith is the number of data transmitted per time unit. The linear communication model therefore represents the duration of a transfer by the linear function _T_ = _β_ \+ _Lτ_ , where _β_ is the latency, or initialization time, incompressible, _L_ the length of the message, and _τ_ the inverse of the bandwidth, i.e. the time required to transfer a unit of data.

2 We, however, expect the evolutionary algorithms to provide best results, but the time required for their computation then prevents any use "on the fly".

3 The dilation of an edge of the source graph is the length of the shortest path, within the target graph, between target vertices onto which are mapped the two source vertices that are the ends of the edge.

4 The notion of "target graph" will be explained further, when we present the "task interaction graphs".

5 The average congestion of a mapping is defined as the sum, for all the edges of the target graph T , of the number of chains _ρ S,T_ ( _e S_) passing through each of them, divided by | _E_ ( _T_ )|. It is therefore proportional to _f C_ ( _τ S,T_ , _ρ S,T_ ), which counts the same chain segments, but per edge of the source graph and not per edge of the target graph.

6 In terms of representation, communication buses will appear as cliques, as each of the elements connected with the bus can communicate similarly with all its peers.

7 Like the path dug by the head of a worm and followed by other segments of its body, hence the name.

8 In the reduction process, considering the information associated with the free variables of a problem often requires an in-depth knowledge of the latter. When the reduction only considers bound variables, we would rather discuss "backtracking".

9 The _diameter_ of a connected graph is the greatest distance measured over the set of shortest paths between all the pairs of graph vertices. Thus, in a graph, there cannot be two vertices located at a distance greater than the diameter of this graph. A _peripheral_ vertex is a vertex located at a diameter distance from another vertex of the graph. A _pseudo-peripheral_ vertex is a potentially peripheral vertex, found by an approximation method faster than exact methods.

10 Improving the density of the parts, under this norm, is similar to the reduction of their _aspect ratio_ in the case of graph partitioning, see index.

11 The quotient graph generated by a partition of a graph is the graph that has as many vertices as there are parts in the partition, so that there is an edge between two of these vertices if there are at least two vertices of the initial graph connected by an edge and each one belongs to one of both parts. In the case of valuated graphs, the weight of the vertices of the quotient graph is equal to the sum of the weights of the vertices in each of the parts, and the weight of the edges of the quotient graph is the sum of the weights of the edges of the initial graph connecting vertices belonging to both parts.

12 On a hypercube of dimension _d_ , whose vertices are numbered from 0 to 2 _d_ − 1, the distance between two given vertices, called _Hamming distance_ , is the number of bits from which the binary representations of both vertex numbers differs (see Figure 5.1).

13 The cocycle of a subgraph is the set of edges of the initial graph where exactly one end belongs to the subgraph, i.e. the set of edges connecting the subgraph to the rest of the graph.

14 SCOTCH: Static mapping, graph partitioning, and sparse matrix block ordering package, www.labri.fr/~pelegrin/scotch/.
PART 2

Optimization Methods for Graph Partitioning

# Chapter 6

# Local Metaheuristics and Graph Partitioning 1

This chapter and the following chapter are dedicated to the application of metaheuristics to the graph partitioning optimization problems. _S_ tate-of-the-art methods are introduced, as well as those which currently appear to be the most efficient and/or promising. _S_ ince numerous adaptations of metaheuristics to graph optimization problems have been proposed in the last 20 years, it has not been possible to condense all of them into a single chapter.

This first chapter focuses on the adaptation of local search metaheuristics. These metaheurstics are characterized, as their name suggests, by the use of a local search method, which is recursively applied to a solution of the problem, so as to improve it. Many local search metaheuristics are stochastic local search algorithms. This implies that iterative algorithms go from one solution to another in the search space, until a given stop criterion is satisfied. These algorithms use heuristics to proceed go from one solution to another, and/or refine these solutions. Among the most well known local search metaheuristics, we can name a few: simulated annealing, tabu search, GRASP (Greedy Randomized Adaptive Search Procedure), variable neighborhood search (VNS) and iterated local search (ILS). For more information on stochastic local search algorithms, refer to [HOO 04].

This chapter is organized as follows: section 6.1 briefly presents what metaheuristics are. Because it has been greatly applied to graph partitioning problems, the simulated annealing metaheuristics and several of its adaptations are described in section 6.2. We have considered it important to introduce in section 6.3 adaptations of iterated local search to graph partitioning, both for the simplicity of this method, and also for the quality of the results found. At the end of this chapter, in section 6.4 we review several references for the adaptation of other local search heuristics and metaheuristics (see section 6.4).

# 6.1. General introduction to metaheuristics

Metaheuristics form a family of optimization alogrithms first appearing in the 1960s to resolve difficult optimization problems. There are a large number of different types of metaheuristics, which range from local search to complex global search algorithms. The term metaheuristic has evolved from the concatenation of meta and heuristic, which are words of Greek etymology whose concatenation signifies: higher-level than heuristic (heuristic meaning to find). Therefore, metaheuristics have in common the use of a high-level abstraction which allows them mostly to adapt to a large range of different problems.

Metaheuristics have a stochastic nature, which allows them to explore a very large solution space more easily. The calculation of the gradient of the objective function is not necessary for these methods. They are often inspired by analogies of nature. The primary qualities of these methods are their adaptability to different problems and their performance in terms of solutions found. Unfortunately, a lengthy adjustment of their parameters is often necessary in order to obtain efficient results, and they are often more costly in terms of calculation time, as compared to dedicated resolution algorithms.

The basic principle of a metaheuristic is to cover the solution space searching for the global minimum by using mechanisms allowing it to come out of local minima of the _energetic landscape_ (term used by extension of the analogy made for simulated annealing in the field of physics). Metaheuristics techniques progress iteratively, by alternating learning, intensification and diversification phases. However, contrary to intensification and diversification, learning is not a mechanism that is common to all metaheuristics techniques. Diversification consists of extending the exploration search area. Inversely, intensification aims to extract from a reduced-size search area, the most interesting points. These two notions, intensification and diversification are paramount in terms of the basic principles of metaheuristics. It is the delicate association of these two concepts, which allows to find a good solution (or even the optimal solution) to the problem. Learning allows to either avoid areas of the solution space which have already been considered as non-interesting, or to recognize areas of strong potential, or to combine both.

Metaheuristics are particulary adaptable to new objective functions. When an optimization problem has few constraints, the objective function used by the metaheuristics is often that of the problem. However, when the problem has several constraints or some are non-linear, a classic method of integrating these constraints is to introduce them into the objective function, which will take into account the violation of constraints by penalizing the solutions which breach them. Another more popular method of resolving this type of problem is to use a multiobjective approach [COL 02].

It is very difficult to classify metaheuristics [BLU 03]. One possibility is to distribute them among two large families [DRÉ 03], _neighborhood_ -based and _population_ -based. Neighborhood metaheuristics have in common an acceptance mechanism of degraded solutions, which allow them to remove themselves from local minima. Those which are population based dispose of a collective mechanism for coming out of local minima.

This chapter will only introduce a few metaheuristics techniques by studying their applications to graph partitioning. For a more detailed presentation on metaheuristics, refer to [DRÉ 03, TAL 09].

Even though rarely considered as a metaheuristic technique, the multilevel method can be considered as part of these methods (for more details on this method, refer to Chapter 2). This is certainly related to the fact that the multilevel method is almost exculsively applied to graph partitioning. However, studies have shown that it could be used as a metaheuristic technique applied to other problems [WAL 04], such as the traveling salesman problem [WAL 01b, WAL 02] or the graph coloring problem [WAL 01a]. However, as we will see, the most efficient adaptations of metaheuristics techniques to graph partitioning optimization problems are often hybrid methods, which use classical metaheuristics techniques with a multilevel algorithm.

# 6.2. _S_ imulated annealing

Simulated annealing is a metaheuristic technique that was created at the beginning of the 1980s, and which was inspired by certain principles of metallurgy. More particularly, simulated annealing has derived its name from the process of alternating between cooling and heating of a material. This process has the effect of minimizing the energy of the material. This metaheuristic technique was introduced by _S_. Kirkpatrick, C. Gelatt and M. Vecchi [KIR 83].

Simulated annealing is now a tried and tested technique, which can be simply put into space, simple to put into place, and which often gives interesting results in a short programming time. Therefore, it is a very interesting method to implement first, before the use of more sophisticated methods, if there is a need.

In the surrounding graph partitioning literature, simulated annealing has been used as much as a tool of direct graph partitioning [JOH 89, WIL 91], as a tool of partition refinement [MAR 96, DIE 96, BAÑ 03, GIL 06].

We introduce the simulated annealing metaheuristic technique in the first section (section 6.2.1), then we will describe a classic adaptation of this method in the case of graph partitioning in the next section (section 6.2.2).

## 6.2.1. _Description of the simulated annealing algorithm_

Simulated annealing is inspired from the experimental technique of annealing used in metallurgy. This allows us to obtain a stable state of the metal, meaning with a minimal energy state. This state is obtained when the material has found a crystalline structure. On the contrary, the well known technique of quenching freezes the material in a metastable state, which implies a state corresponding to a local minimum in energy. The technique of annealing is used in order to allow the material to be in a stable state and avoid the metastable state corresponding to the local energy minimum.

In order to avoid the material remaining in a metastable state, the technique of annealing consists of raising the temperature of the material, then bringing it back down slowly. In computing, the method of simulated annealing transposes this technique to the resolution of an optimization problem. Therefore, the temperature, essential parameter of physical annealing, is directly used in the method of simulated annealing as the control parameter. In the same way, by analogy with the physical processes, the energy of the system becomes the cost function to be minimized. In order to simulate the thermodynamic evolution of the system, simulated annealing uses the Metropolis acceptance rule [MET 53]. The latter is based on the Boltzmann distribution, which is used in physics to determine the distribution of particles depending on their energy level. In practical terms, it allows to choose a new element in relation to a previous one, either because it has less energy, or has a certain probability, which depends on the energy difference between the two elements.

The simulated annealing algorithm is described in numerous works [SIA 89, DRÉ 03]. Its principle is simple: going from any inital configuration, a new state is created from the previous state by an elementary modification of this state. It is accepted if its energy is lower, if not it is accepted with a certain probablity. This pattern is repeated by replacing the initial state by the state accepted, as long as thermodynamic equilibrium is not achieved. When this equilibrium is attained, the temperature is reduced. Then the thermodynamic equilibria are renewed, as long as the system is not frozen. There are different variants of this method [ING 89].

This method, therefore allows one to accept a degraded state in relation to the previous state. This particular nature, common to all metaheuristics techniques, allows exploration of a larger part of the solution space and avoidance of local minima.

Algorithm 6.1 introduces the method of simulated annealing technique for the minimization of a combinatorial optimization problem of with an objective function called _energy_ function and a solution space _E_. The initial state einit can be randomly chosen in the solution space, or can be the result of another heuristic technique. This last solution does not necessarily result in time or efficiency gain, especially if the heuristic found a local minimum from which it has trouble coming out. The initial temperature of the system, _T max_, corresponds to the maximum temperature, and therefore the

* * *

**Algorithm 6.1.** _S_ imulated annealing algorithm

* * *

* * *

temperature of the disturbed state of the system. A classical stop criterion / consists of iterating the algorithm until this temperature falls below a certain threshold.

The algorithm of simulated annealing is based around two loops: an internal loop and an external loop. The internal loop iterates as long as the system is not stable. The stabilization criteria of the system vary widely depending on the simulated annealing algorithms. In numerous cases, the system is said to be stable, when a predetermined number of consecutive elementary modifications have been rejected by the Metropolis acceptance rule. The external loop reduces the temperature, in order to simulate annealing when the system is stable.

The internal loop creates a new state by elementary modification of the previous one, then applies to it the Metropolis acceptance rule. The Boltzmann function: _R_ → [0; 1] can theoretically represent any probability density of a probability law. However, in order to comply with the origins of the method and through simplicity of calculation, the Boltzmann factor, which is deprived of the Boltzmann constant, is almost always used for this function:

Therefore, when the energy difference is positive, the lower this is, the higher the chance that the new state will be accepted. The temperature also plays an important role in the probability of acceptance. The higher it is, the higher the chance of selecting the new state.

The best state found by the algorithm is conserved and successively updated by the interal loop. It is this state which is returned when the external loop, and therefore the annealing, ends.

The external loop, i.e. the loop of temperature decrease, brings into play the internal loop, as well as the _reducing_ function, and is responsible for decreasing the temperature. Numerous temperature decrease laws can be used. One of the most widespread and also simplest, is the linear decrease law:

The value of the parameter _T f act_ is mostly between 0.9 and 0.99.

In order for the simulated annealing algorithm to be efficient, an elementary modification of _e_ must create a state _e′_ that is very near to _e_. With this aim in view, the elementary modification of _e_ must be local and unique. In the case where the new element _e′_ is too far away from e in the solution space, then the annealing will simply behave as a random search algorithm.

## 6.2.2. _Adaptation of simulated annealing to the graph bisection problem_

We introduce the adaptation of simulated annealing for graph partitioning described in the articles of D. Johnson, C. Aragon, L. McGeoch and C. _S_ chevon [JOH 89], in this section. More precisely, this concerns an adaptation of simulated annealing to graph bisectioning of unweighted graphs. As we will see in the following section, this adaptation has been extended in [BIC 07] to _k_ -partitioning of weighted graphs.

The problem of partitioning covered in [JOH 89] relates to the problem of constrained partitioning (this problem is described in Chapter 1, page 13). That is to say, it consists of finding a graph bisection which minimizes the cut of the partition, by respecting a maximum partitioning balance, _bal_max_. More specifically, the algorithm of simulated annealing, which was introduced in [JOH 89] conducts searches to find a partition _P_ 2 of a graph _G_ = ( _V, E_ ) in two parts _V_ 1 and _V_ 2 of maximum size  .

As we have seen in the previous section, simulated annealing is based on the notion of neighborhood, and more exactly movement from one state to a neighboring state. In this adaptation to graph partitioning, we will consider that two partitions are neighbors if we can go from one to the other by moving only one vertex of one section to another section. By using the previous notations, for every vertex v ∈ V1, we will consider that _P_ 2 = ( _V_ 1, _V_ 2) and   = ( _V_ 1 − { _v_ }, _V_ 2 ∪ { _v_ }) are two neighboring partitions (the same for _v_ ∈ _V_ 2).

For the purpose of respecting the constraint on the partitioning balance, the cost function used in [JOH 89] is the following:

[6.1]

where the _size_ function returns the number of elements of a set and where _α_ is a constant in charge of penalizing the partition when it no longer respects the partitioning balance _bal_max_.

As the authors have noted, this cost function allows acceptance of bisections, which do not respect the constraint on the partitioning balance. However, at low temperatures and if α is correctly chosen, the part 2 of this function forces the solutions to be almost perfectly balanced. This is a penalty function, which is quite often used in the cases of adaptations to simulated annealing. The interest of this penalty function is that it allows one to escape from a local minimum by going through unauthorized states, and this can be much easier than strictly respecting the constraint. The risk, of course, involves an inability to return to an acceptable state in terms of constraints, which implies a perfectly weighted bisection. This risk remains low in the case of small graphs (less than 1,000 vertices), but increases with the size of the graph and the number of parts ( _k_ > 2).

In the case where the final partition returned by the simulated annealing is not correctly weighted, a heuristic technique is used to re-establish the balance of the partition. This heuristic technique consists of moving a vertex of the largest part of the bisection toward the smallest, as long as the two parts are not of the same size. The choice of the vertex moved is not random, this vertex is chosen to be the vertex which minimally increases the cut cost of the new partition (or decreases it the maximum if possible!). Even though authors do not mention it, such a heuristic technique is recommended for using the Fiduccia-Mattheyses data structure (this structure is decribed in Chapter 2, page 46), in order to choose, as quickly as possible, the vertex with the highest gain (meaning the vertex which decreases the cut cost the maximum or, if this is not possible, the one which increases it the least).

Algorithm 6.2 shows the adaptation of simulated annealing to graph partitioning and is described in [JOH 89]. The working principle of this algorithm is identical to that of the classical simulated annealing Algorithm 6.1 which is described in the previous section (section 6.2.1). However there exists a slight difference between these two algorithms. This difference lies in the external loop. The stop criterion of the algorithm uses a counter, _stop_counter_ , which allows to know if the state of the system is frozen or not. This variable is increased each time that an internal loop has not had enough accepted movements. More precisely, _MinPerct_ corresponds to the minimum percentage of movements accepted over an internal loop in order to not raise _stop_counter_. When _stop_counter_ is equal to 5, the system is said to be frozen, and the algorithm stops and returns the best bisection found.

* * *

**Algorithm 6.2.** Adaptation of simulated annealing to graph bisectioning shown in [JOH 89]

* * *

* * *

Algorithm 6.2 considers 4 parameters as arguments, in addition to the graph to partition: _T max_, _SizeFact_ , _TempFact_ and _MinPerct_. We have just seen the usefulness of the _MinPerct_ parameter. The _TempFact_ parameter corresponds exactly to the _T fact_ parameter described in the previous section. Thus, _TempFact_ is the factor by which the temperature is linearly decreased each time the system is judged to be sufficiently stable. The _SizeFact_ parameter allows to modify the number of iterations of the stabilization loop. Finally, and contrary to what is often carried out, _T max_ is not set manually by the user. It is determined after a series of iterations of the stabilization loop with different temperature values. _T max_ is equal to the value of the temperature, which allows to find the percentage _InitProb_ of movements accepted at the end of the stabilization loop. Therefore, the value _T max_ is set depending on the percentage _InitProb_ of movements initally accepted.

In the end, the adaptation of simulated annealing, as shown in [JOH 89] requires setting up of 5 parameters: _α_ (equation [6.1]), _InitProb_ , _SizeFact_ , _TempFact_ and _MinPerct_. By way of example and in order to set orders of magnitude, the authors have judged the following values as satisfactory for their experimentations on graphs, which have between 100 and 1,000 vertices: _α_ = 0.05, _InitProb_ = 0.4, _SizeFact_ = 16, _TempFact_ = 0.95 and _MinPerct_ = 2%. After implementation, the execution time of Algorithm 6.2 is slower than those described in the next section, even if the latter partition the graph into 55 parts instead of 2. The adaptation of this simulated annealing algorithm to _k_ -partitioning in _k_ parts is described below.

In JOH 89] this adaptation of simulated annealing seems to give results similar to those of the Kernighan-Lin ([section 2.5.2) algorithm for graphs between 100 to 1,000 vertices, but with a longer execution time (differential of 1 to around 12).

## 6.2.3. _Generalizing this adaptation to k-partitioning_

The adaptation of simulated annealing to graph bisectioning described in the previous section was extended in [BIC 07] to _k_ -partitioning of weighted graphs. We will now describe this generalization.

In the previous section, the optimization problem consisted of finding a graph bisection which minimizes the cut, for the constraint of finding the same size for both parts of the bisection. The objective function of the equation [6.1] allowed us to respond to this problem. Generally, the weighted graph partitioning problem consists of finding a partition _P k_ = { _V_ 1,..., _V k_} minimizing an objective function _f_ and with constaint a maximum for constraint a maximum weight for the heaviest part of the graph. A new cost function, which responds to this problem can be proposed:

In this equation, _w_ ( _V i_) is equal to the sum of the weights of the vertices of the set of vertices _V i_ and _α_ remains, as in the previous section, a parameter of the algorithm. This parameter allows to control the balance of the sought part of the partition.

Another possible cost function, which is closer to the equation [6.1], would be:

where   represents the maximum weight of a part which allows to satisfy the constraint of partitioning balance _bal_max_.

In order to proceed from graph bisectioning to _k_ -partitioning, we have seen in Chapter 1, page 17, that there are two methods: recursive bisectioning or direct _k_ -partitioning. The method of recursive bisectioning is described there in detail; we will therefore not cover it here. Going to direct _k_ -partitioning is quite simple to implement in the case of simulated annealing adaptation. As we have seen, simulated annealing works by neighboring between two solutions. Thus, in the case of bisectioning, the neighbor of a partition is the partition where one vertex has changed part. It is the same for k-partitioning, only in this case a vertex can move to _k_ − 1 parts, instead of one single part. The neighborhood is therefore larger! The modifications to be made to Algorithm 6.2 to integrate this adaptation to _k_ -partitioning are therefore minimal, and hence we will not rewrite this new algorithm.

## 6.2.4. _Assessment of simulated annealing adaptation to graph partitioning_

The adaptation of simulated annealing to the graph partitioning optimization problem is quite simple to implement, as we have seen in the previous sections. One of the biggest advantages of simulated annealing is its adaptability to different objective functions and constraints of graph partitioning optimization problems. In addition, simulated annealing allows to carry out direct _k_ -partitioning, which is even more interesting, when we know that certain solutions cannot be found by the recursive bisection method [SIM 97].

However, these advantages cannot cover up the strong deficiencies of these adaptations. In effect, these adaptations are very slow in comparison to multilevel methods (for more details on this method, see Chapter 2), which take less than a second of computation time, whereas simulated annealing algorithms require several minutes before finding interesting solutions. In addition, the solutions found by these adaptations of simulated annealing are of the same quality, as those found by the Kernighan-Lin algorithm, but those are clearly worse than those of the multilevel method.

Therefore, simulated annealing can be used favorably on small-sized partitioning problems, whose objective function and constraints are non-traditional or even exotic. It is in these conditions that the advantages of simulated annealing can be fully exploited for the graph partitioning problem. Through the resolution of more traditional and larger partitioning problems, it would be preferable to foresee the usage of another approach.

Finally, we should also note that simulated annealing used in conjunction with a tabu search hybridized with a multilevel method gave very good results in [BAÑ 03, BAÑ 04]. As we will see later, numerous classic metaheuristics obtain better results, as much in terms of computation time as the quality of results found, when they are used in conjunction with a multilevel approach.

# 6.3. Iterated local search

Iterated local search (abbreviated to ILS) is an exploration local search metaheuristic. This means that ILS will explore the solution space of the problem by using an internal local search method.

If metaheuristics techniques have a tendency to become more and more complex with time in order to obtain the best results, the iterated local search remains for its part simple and functional.

## 6.3.1. _Presentation of iterated local search_

In the resolution process of an optimization problem, the first approach consists of using a local search method, for example a heuristic dedicated to the resolution of the problem. This essential approach, since it is sufficient in many cases, however does not often allow resolution of difficult problems, especially when they are of a large size. It is this acknowledgment which leads users of metaheuristics to call upon these methods.

The iterated local search metaheuristic is based on the local search method, which is initially used to look for a response to the following question: can we find better solutions by iteratively creating a sequence of solutions generated by a local search algorithm, rather than by carrying out different random executions of this algorithm?

The answer given by the iterated local search is yes, but under certain conditions. For this, the local search algorithm must not be incompatible with the iterated search.

Let's consider an optimization problem with _f_ the objective function (or cost function) to minimize, and _S_ its solution space. In order to work efficiently, a local search algorithm uses the notion of neighborhood of the topological structure of _S_ (for this it is necessary that _S_ is not a characterless set, but possesses certain properties, which is in general true for optimization problems). By assuming that the local search algorithm is deterministic, this algorithm can be seen as a surjective function of _S_ toward _S_ *, which possesses the following property: ∀ _s_ ∈ _S_ , ∃! _s_ * ∈ _S_ *, _f_ ( _s_ ) ≤ _f_ ( _s_ *). We therefore, have a projection of the set of solutions _S_ of the problem onto a smaller set _S_ * of locally optimal solutions. For a locally optimum solution _s_ *, there are several solutions _s_. We will say that these solutions are part of the same basin of attraction (also called energy sink).

Having a local search algorithm and therefore a surjection of _S_ onto _S_ *, we will search to find the best possible usage of this algorithm. That is to say, we will search to find, as easily and quickly as possible, the global optimum by considering the local search algorithm as a black box which we will name _LocalSearch_.

### 6.3.1.1. _Random search_

Considering the optimization described above, the simplest method to solve it iteratively consists of repeating the local search heuristic from different points chosen randomly in the solution space _S_. This process called _random restart_ allows to find solutions _s_ * ∈ _S_ * independent of each other. However, we observe that the distribution of _S_ * becomes more and more acute around a mean value as _S_ increases. This means that the relative fluctuations of _s_ * around a mean cost _s_ * approach zero when the size of _S_ tends to infinity. In more practical terms, this means that as the size of the problem increases, the more difficult it becomes to find an interesting solution with a random search. Therefore, the number of interesting solutions increases much less rapidly than the number of average solutions when the size of the problem increases. In [SCH 99], G.R. _S_ chreiber and O.C. Martin verify these observations in the case of graph bisectioning. They also observe that the mean cost _s_ * is a fixed percentage above the optimum cost. Therefore, random restart is only interesting in the case of small-sized problems.

### 6.3.1.2. _Iterated local search_

The problem encountered by random search is simply summarized by the fact that, when _S_ increases, _S_ * also increases and the probability of acquiring a good element in _S_ * decreases! It therefore becomes interesting to use a strategy that allows the judicious exploration of _S_ *. In the same way that the local search used the notion of neighborhood in _S_ , the iterated local search algorithm will consider using a notion of neighborhood in _S_ *.

The strategy of the iterated local search is described below. Let's assume that the current solution of the iterated local search algorithm is _s_ * ∈ _S_ *. A disturbance is applied to _s_ * which generates the element _s_ ′ ∈ _S_. The local search algorithm _LocalSearch_ is applied to _s_ ′ and returns the solution _s_ *′ ∈ _S_ *. If the solution _s_ *′ fulfills certain conditions, for example _f_ ( _s_ *′) < _f_ ( _s_ *), then the strategy is applied to _s_ *′, if not it is again applied to _s_ *.

In this strategy, the notion of neighborhood in _S_ * is never directly used. However, in order for this method to be effective and different from a simple random search, it is necessary that the disturbance applied returns a solution _s_ *′ close to _s_ *. By returning to the solution space _S_ , we will say that _s_ * and _s_ *′ are two close basins of attraction (energy sinks). In the case where the disturbances are too strong, the iterated local search returns to a random search. In the case where the disturbances are too weak, _LocalSearch_ always returns to _s_ * and the search restricted in the basin of attraction. Therefore, the iterated local search uses a notion of neighborhood in _S_ * which is not strict, but induced by the notion of neighborhood in _S_.

The iterated local search uses the two main basic principles of a metaheuristic technique (section 6.1): intensification and diversification. It is the local search algorithm _LocalSearch_ which ensures the intensification phase, and the disturbance ensures the diversification phase. We will see that learning (third phase, optional, completing the two previous fundamental phases) can also be introduced into the iterated local search algorithm.

* * *

**Algorithm 6.3.** General iterated local search algorithm (ILS)

* * *

1: **procedure** ILS

2: _s_ 0 ← _CreateInitialSolution_

3: _s_ * ← _LocalSearch_ ( _s_ 0)

4: **repeat**

5: _s_ ′ ← _Disturbance_ ( _s_ *, _historic_ )

6: _s_ *′ ← _LocalSearch_ ( _s_ ′)

7: _s_ * ← _AcceptanceCriterion_ ( _s_ *, _s_ *′, _historic_ )

8: **until** Stop criterion

9: **end procedure**

* * *

A generic version of the iterated local search algorithm is given by Algorithm 6.3. Its efficiency is based on three procedures, in addition to _LocalSearch_ which of course forms the central procedure of the ILS algorithm: _CreateInitialSolution_ , Disturbance and _AcceptanceCriterion_. We shall quickly review each of these procedures to specify their role and describe their influence.

_CreateInitialSolution_ allows, as its name suggests, to find an initial first solution _s_ ∈ _S_ to the problem. If this solution can be randomly generated, we will prefer sometimes to use a greedy approach. In fact, since a greedy algorithm allows to find better quality solutions as compared to random methods, the ILS algorithm will reach quality solutions more rapidly and potentially a higher level of quality. On the other hand, this will also allow us to better exploit the performances of local search algorithms, which are, in general, more efficient in improving a solution generated by a greedy approach, as compared to improving a random solution.

_Distrubance_ is undoubtedly the most critical function in the ILS algorithm. As we have seen above, it is due to this that the iterated local search differs from a random search. However, for this a good setup is required, which is more or less long and tricky and depends on the problems. One way of improving this procedure is, if we can, is to make it find a disturbed solution by considering the history of solutions _s_ *. This way, we can introduce the learning functionality into the iterated local search.

There are several types of disturbance functions. The simplest one consists of modifying a determined number of parameters of the standard solution _s_ * to create _s_ ′. Of course, another possibility consists of modifying this number during the course of the ILS algorithm iteration. We speak of adaptive disturbance. A more complex way of disturbing _s_ * consists of changing certain parameters of the problem and proposing a new solution, which responds to these new parameters. For example, in the case of graph partitioning, we can add or remove graph edges. We will see in more detail an example in section 6.3.3. Finally, the disturbance function influences the behavior of other procedures, which are related to the ILS algorithm. This is therefore without doubt the trickiest function to choose and will be decisive for this algorithm, apart from _LocalSearch_.

The _AcceptanceCriterion_ function allows to choose the solution _s_ * which is to be used during the next ILS iteration. This procedure allows to adjust the balance between intensification ( _s_ *, found by _LocalSearch_ ) and diversification ( _s_ *′, result of _Disturbance_ ). Its simplest implementation consists of returning the solution of minimum cost. However, it is also possible to still accept _s_ *′ or even use the Metrolpolis acceptance rule (also used in simulated annealing, section 6.2).

The iterated local search algorithm, if it is well parameterized, is both more efficient in terms of solutions found and in terms of calculation time as compared with a random search technique. This second property is due to the fact that, when a correctly parameterized permutation is used, the solution _s_ *′ conserves certain local configurations of _s_ * and its refinement by _LocalSearch_ , and is therefore less costly than that of a random solution. For a more in depth description of iterated local search and notably to study its application to problems other than graph partitioning, refer to [LOR 03].

## 6.3.2. _Simple adaptation of iterated local search to graph partitioning_

In this section, we propose a simple adaptation, which is however, efficient for certain graphs, of iterated local search to the problem of graph bisectioning from [BIC 09]. More precisely, we will study the problem of cut minimization with strong constraints on the size of the sections, meaning that both parts of the bisection must be of the same size.

### 6.3.2.1. _Adaptation_

The objective of this simple adaptation, named SBILS ( _Simple Bisection Iterated Local Search_ ), is to show that the iterated local search can be truly more efficient than a simple random search, in the case of bisectioning large graphs. For this, we used the most commonly used local search algorithm in graph partitioning, a Kernighan-Lin type refinement algorithm, which is extremely efficient for bisections. This algorithm is shown in Chapter 2, section 2.5.2. In order to have faster computation times, the number of passes of the Kernighan-Lin algorithm is limited to _MaxIterKL_ passes. We observe that by doing so we do not decrease the performances of the Kernighan- Lin algorithm. A pass consists of permuting two sets of vertices of different parts in a partition. This implies that a bisection involves _P_ 2 = ( _V_ 1, _V_ 2) a pass that consists of permuting two subsets of vertices of the same size, _S_ 1 ⊂ _V_ 1 and _S_ 2 ⊂ _V_ 2, in order to obtain a partition (( _V_ 1 ∩ _S_ 1) ∪ _S_ 2, ( _V_ 2 ∩ _S_ 2) ∪ _S_ 1). If an iteration of the Kernighan-Lin algorithm does not allow improvement of the current solution standard, the local search stops and returns this solution.

As we have seen in the previous section (section 6.3.1), once the local search algorithm is chosen, there remains three procedures to determine: _CreateInitialSolutions_ , _Disturbance_ and _AcceptanceCriterion_ , as well as the stop criterion of the algorithm. The latter is the maximum number of iterations without improvement of the solution ( _MaxIter_ ).

In this simple implementation, _CreateInitialSolutions_ consists of randomly creating a balanced bisection, which therefore implies that both parts must be of the same size. A graph growing greedy algorithm such as GGP or GGGP (these algorithms are introduced in Chapter 2, section 2.4.2), can be used.

The _AcceptanceCriterion_ procedure simply consists of conserving the minimum cost bisection, for the next iteration of SBILS. More strictly,

As we have discussed in the previous section, finding an adapted disturbance is the most difficult part. Several trials have therefore been carried out. In its most standard form, a disturbance consists of modifying a predetermined number of parameters of the standard solution. Adapted to graph bisectioning, this mechanism consists of moving some vertices from a part of the partition to another. Let _NbDep_ be this number of vertices. Returning to the case of a bisection _P_ 2 = ( _V_ 1, _V_ 2), this mechanism consists of moving a subset _S_ 1 ⊂ _V_ 1 of _NbDep_ vertices from _V_ 1 to _V_ 2 to obtain the bisection ( _V_ 1 ∩ _S_ 1, _V_ 2 ∪ _S_ 1). The results obtained for a fixed number and a variable number of movements have been compared.

Several fixed values of _NbDep_ have been tested. These tests have been carried out on two categories of graphs: almost regular graphs (see Table 6.1) and ordinary/simple graphs (see Table 6.2). It has been noted that, for _NbDep_ > 1% _n_ (respectively _NbDep_ > 10% _n_ for regular graphs), with n the number of graph vertices, the number of iterations of the Kernighan-Lin algorithm should be significant (around 4 to 5). For a value of _NbDep_ ≤ 1%n (N bDep ≤ 10%n respectively), an iteration of the Kernighan-Lin algorithm was enough for the large majority of cases. Finally, with too small a value of _NbDep_ , the solutions found were always the same. This initial experimental study allowed us to conclude that _NbDep_ = 1%n (N bDep = 10%n respectively) and _MaxIterKL_ = 3 allowed us to approach an interesting neighborhood on _S_ * and that the value of _NbDep_ depended on the nature of the graphs.

Table 6.1. Comparison on regular graphs between SBILS, random search iterations using the Kernighan-Lin refinement algorithm(RA KL), METIS and SCOTCH

Table 6.2. Comparison on ordinary/simple graphs between SBILS, random search iteration using the Kernighan-Lin refinement algorithm (RA KL), METIS and SCOTCH

Inspired by the variable neighborhood search [HAN 01, MLA 97], a variable value of _NbDep_ was found. During the iteration of the SBILS algorithm, _NbDep_ is multiplied by a factor that is considered as a parameter (between 1 and 2), if a lower cut solution is not found. In the case where such a solution is found, _NbDep_ is restarted at 1%n. However, this approach has been found to be less efficient than the one that uses a fixed value of _NbDep_.

### 6.3.2.2. _Evaluation_

Tables 6.1 and 6.2 show a comparison of partitions found by the SBILS algorithm which was just shown, a random search using the Kernighan-Lin algorithm (RA KL) and graph partitioning software SCOTCH and METIS. The description of the graphs used can be found in the Appendix. These graphs can be downloaded from the Internet site of the book1.

In order to test the algorithms several times on the same graphs, permutations of the numbering of these have been carried out. This allows to obtain identical graphs and therefore identical cuts, but emulates a different random seed generator every time, without having to touch the source code of the algorithms.

The results shown for the two partitioning tools SCOTCH and METIS correspond to the minimum cuts found after 10 minutes of iterations of these algorithms, on permutations of the original graphs, or 20 iterations if the time was too short. For each graph, the random search applied the Kernighan-Lin algorithm with _MaxIterKL_ = 3 over 1,000 permutations. The minimum and average cuts are reproduced, as well as the standard deviation. In addition, the SBILS algorithm was applied to 10 permutations of each ordinary graph and 5 permutations of each regular graph. In the case of the latter, _NbDep_ = 1%n and for the others, _NbDep_ = 10%n. In both cases, _MaxIter_ = 100 and _MaxIterKL_ = 3.

In the case of regular graphs (Table 6.1), the partitions found by the SBILS algorithm are almost always equal to the minimum cut (which is know for these graphs). They are also of better quality than those found by SCOTCH and METIS software, which are above the minimums cuts. In addition, these results clearly show that the SBILS algorithm definately does not carry out random searches. In terms of execution time, the Kernighan-Lin algorithm is 48 to 69 times faster than the SBILS algorithm (last column in the table). We must, of course, take into consideration the fact that SBILS carries out at least 100 iterations.

In the case of ordinary graphs (Table 6.2), the results of SBILS are not as good. The average cuts of SBILS are not as good as those (minimum) of METIS and equivalent to those of _S_ cotch. On the other hand, for several graphs, the results of SBILS are close to that of a random search. In fact, for these graphs whose topology distinctly differs from one to another, a personalized setup is more efficient. Therefore, with a different value of _NbDep_ , certain results can be improved.

We can deduce from these results that this simple adaptation of a local search algorithm is efficient, but depends on the setup of the algorithm. After having adapted the iterated local search metaheuristic, with a local search heuristic the Kernighan-Lin refinement algorithm, is interesting to test with a more efficient multilevel approach.

## 6.3.3. _Iterated local search and multilevel method_

The multilevel method is both very efficient and extremely quick to solve the graph partitioning optimization problem (Chapter 2 explains this method in detail). This method is made up of three phases: contraction, partitioning and refinement. Considering its speed (less than a second of computation time for a graph with several tens of thousands of vertices), it is natural that we should try it out to to improve its performances by succesive iterations. In this perspective, the multilevel method, even if it ressembles a metaheuristic, will act as a local search algorithm for the iterated local search.

Even though not refered to works on iterated local search, an ILS algorithm and the multilevel approach is described in [WAL 04, WAL 08]. A slightly modified version of this algorithm, named _Iterated Multilevel Partitioning_ (IMP), is described below.

As we have seen, the efficiency of iterated local search depends on the quality of local search and the distrubance used. The disturbance can either modify the parameters of the running solution, or modify the parameters of the problem to create a new solution. It is this last possibility which will be used by the IMP algorithm.

Let's consider an iteration of the ILS algorithm,   is the standard partition of the graph G. The disturbance involves progressively contracting the graph G in such a way that at each level of contraction, a vertex can only assemble with vertices of the same part in   . This stage corresponds to that of the disturbance of the ILS algorithm. In fact, we can consider that this stage consists of creating a new graph _G_ ′ image of _G_ from which all the edges linking the parts of   will be removed. The algorithm continues through stages of partitioning and refinement of the multilevel method. These are done on the successive contractions of _G_ , but this time taking into account all the edges present in _G_. These two stages return a partition   of _G_. Finally, the IMP algorithm selects the lowest cut partition before reiterating. The IMP algorithm is also described in Algorithm 6.4. The stop criterion of this algorithm is the maximum number of iterations without improvement of the solution ( _MaxIter_ ).

Therefore, the disturbance function of this algorithm consists of disturbing not the solution   , but the graph _G_ itself, by taking into account   . In order that this algorithm does not always return to the same partition, it is necessary that at least one of the phases of the multilevel algorithm is stochastic.

In [WAL 04], the contraction of the vertices of _G_ ′ is done until a single vertex remains in each of the parts of   . We propose here to stop beforehand, in order to achieve a greater diversification.

* * *

**Algorithm 6.4.** Iterated multilevel partitioning

* * *

1: **procedure** IMP( _G_ = ( _V_ , _E_ ), _MaxIter_ )

2:   ← _Multilevel_ ( _G_ , _k_ )

3: _NbIter_ ← 0

4: **repeat**

5: _G_ ′ ← _G_ deprived of the edges linking the parts of

6: { _G_ 1,..., _G l_} ← _Contractions_ & _Projections_ ( _G_ ′, _G_ )

7:   ← _Partitioning_ & _Refinement_ ({ _G_ 1,..., _G l_})

8: if cut(  ) ≤ _cut_ (  ) **then**

9:   ←

10: _NbIter_ ← 0

11: **else**

12: _NbIter_ ← _NbIter_ \+ 1

13: **end if**

14: **until** _NbIter_ > _MaxIter_

15: **end procedure**

* * *

The results obtained for this algorithm are very good if we trust the Website _Graph Partitioning Archive_ 2. At the end of 2008, practically a third of the 816 best solutions found on this site were found with the algorithm described in [WAL 04]. A comparison of the quality of the solutions found by the IMP algorithm and the multilevel algorithm used as a local search can be interpolated from [WAL 04]. We can refer to the curves which are published there (Figures 2, 3 and 4 for λ = 64) for a graph partitioning in 16 sections with a balance of 3%. The multilevel algorithm finds on average on 16 graphs solutions, 20% worse than the best known, whereas those of the IMP algorithm are situated between 3.2% and 4.2% depending on the values _MaxIter_ (2 _i_ , _i_ = 0,..., 4). As we can see, the improvement brought by iterated local search is substantial in this case.

However, these gains in performance also increase the computation time. The computation time is difficult to determine for such an algorithm due to its iterative nature. However, it is interesting to know orders of magnitude. We can deduce from [WAL 04] that the IMP algorithm takes 15 seconds for _MaxIter_ = 1 and up to 50 seconds for _MaxIter_ = 16 to partition the 4elt graph (sparse graph with 15,606 vertices and 45,878 edges) into 16 sections with a balance of 3%, whereas the multilevel algorithm takes 0.3 seconds. The IMP algorithm is therefore 50 to 500 times longer. Unfortunately, the study does not indicate the number of iterations of the IMP algorithm.

# 6.4. Other local search metaheuristics

The evaluation of several metaheuristics adapted to graph partitioning is proposed in [GAN 98]. A comparison between the adaptation of simulated annealing, tabu search and genetic algorithms is proposed.

In WAL 04, WAL 08], C. Walshaw describes the multilevel method (introduced in [Chapter 2) as a method that allows to improve the performance of metaheuristics. As he points out, the multilevel method cannot function without the use of a heuristic. Therefore, in an iterated form of the type of Iterated Multilevel Partitioning algorithm, the multilevel method complies with the definition of local search metaheuristic. However, few works have used this approach to solve optimization problems outside of graph partitioning.

We saw in section 6.2 that simulated annealing – undoubtedly the most well known stochastic local search algorithm – can be easily adapted to problems of graph partitioning. In this section, we will give a certain number of references on adaptation works to the problem of graph partitioning of different stochastic local search algorithms: greedy algorithm, GRASP, tabu search and ant colonies. We will detail in particular the adaptation of the latter.

## 6.4.1. _Greedy algorithms_

The principle of a greedy algorithm is to construct a solution through successive stages, by choosing the local optimum at each stage, in the hope of obtaining the global optimum at the end of the iteration. This general principle of iteration, which goes through a local optimum, is that of the greedy approach, and can mean that this is a local search metaheuristic. However, this approach is not stochastic, and we will instead discuss greedy heuristics.

Numerous greedy algorithms have been proposed to resolve the problem of graph partitioning. Two of these algorithms are detailed in Chapter 2 on the multilevel method (page 38) since they act during the partitioning stage of this method: the _Graph Growing Partitioning algorithm_ (GGP) [KAR 98] and the _Greedy Graph Growing Partitioning algorithm_ (GGGP) [KAR 98]. However, not all the greedy algorithms for graph partitioning were created to be used by the multilevel method, even since its emergence. The greedy algorithms introduced in [LOU 07, BAT 99, CIA 94] were designed to partition graphs of all sizes directly and efficiently.

The GRASP metaheuristic is built around the use of a greedy algorithm. The iterative method is split into two phases: construction and local search. The construction phase is reserved for a greedy algorithm. For more information on this method, refer to [RES 03]. An adaptation of this method is proposed in [LAG 94].

## 6.4.2. _Tabu search_

The tabu search is a local search metaheuristic based on neighborhood. The algorithm moves from one solution s1 to another neighboring solution _s_ 2. If the new solution _s_ 2 is less efficient than _s_ 1, the next iteration of the algorithm risks repeated finding of _s_ 1 again. In order to avoid this problem and come out of the basins of attraction (energy sinks), a mechanism prohibits a certain number of regressions (a tabu list).

A tabu search algorithm adapted to perfect graph bisectioning is proposed in ROL 96]. It is compared to the Kernighan-Lin refinement algorithm ([Chapter 2, page 43) and simulated annealing of D. Johnson, C. Aragon, L. McGeoch and C. _S_ chevon (section 6.2.2) where it gives interesting results on 149 graphs. Unfortunately, this concerns small graphs of 10 to 500 vertices.

The multilevel algorithm introduced in [BAT 99] uses a method of tabu search during partitioning and refinement phases. This technique allows a greater homogeneity of the partition cuts found than with a traditional multilevel method. This is one of the rare methods to show a multilevel method using a metaheuristic, and not the opposite.

# 6.5. Conclusion

Among the local metaheuristics, the one which has been the most often adapted to the graph partitioning optimization problem is certainly simulated annealing. If the great adaptabilty of this method is a certain advantage for solving unusual problems, its performance is limited in the cases of constraint graph partitioning. In order to solve the constraint graph partitioning problem, another method that has emerged is very interesting to use: iterated local search. _S_ urprisingly, few works mention the use of this metaheuristic to solve the graph partitioning problem, whereas the efficiency of this method is very interesting, especially when it is hybridized with the multilevel approach.

Generally, this chapter shows that local metaheuristics stand to gain by being hydridized with a multilevel algorithm, as much in terms of the quality of the partitions found, as well as for execution times.

We will see in the following chapter (Chapter 7), how to use population based metaheuristics for solving graph partitioning optimization problems.

# 6.6. Bibliography

[BAÑ 03] BAÑOS R., GIL C., ORTEGA J., MONTOYA F., "Multilevel heuristic algorithm for graph partitioning", in _Proceedings of the European Workshop on Evolutionary Computation in Combinatorial Optimization_ , p. 143-153, 2003. 160 Graph Partitioning

[BAÑ 04] BAÑOS R., GIL C., ORTEGA J., MONTOYA F., "A parallel multilevel metaheuristic for graph partitioning", _Journal of Heuristics_ , vol. 10, p. 315-336, 2004.

[BAT 99] BATTITI R., BERTOSSI A.A., "Greedy, prohibition, and reactive heuristics for graph partitioning", _IEEE Transactions on Computers_ , vol. 48, no. 4, p. 361-385, 1999.

[BIC 07] BICHOT C.-E., "A new method, the fusion fission, for the relaxed k-way graph partitioning problem, and comparisons with some multilevel algorithms", _Journal of Mathematical Modeling and Algorithms (JMMA)_ , vol. 6, no. 3, p. 319-344, 2007.

[BIC 09] BICHOT C.-E., "Metaheuristics for graph bisection", in _Proceedings of the 10th ACM Genetic and Evolutionary Computation Conference_ , p. 1801-1802, July, 2009.

[BLU 03] BLUM C., ROLI A., "Metaheuristics in combinatorial optimization: overview and conceptual comparison", _ACM Computing Surveys_ , vol. 35, no. 3, p. 268-308, 2003.

[CIA 94] CIARLET P., LAMOUR F., Recursive partitioning methods and greedy partitioning methods: a comparison on finite element graphs, Report no. CAM report 94-9, UCLA, 1994.

[COL 02] COLLETTE Y., SIARRY P., _Optimisation multiobjectif_ , Eyrolles, 2002.

[DIE 96] DIEKMANN R., LÜLING R., MONIEN B., SPRÄNER C., "Combining helpful sets and parallel simulated annealing for the graph-partitioning problem", _Parallel Algorithms and Applications_ , vol. 8, p. 61-84, 1996.

[DRÉ 03] DRÉO J., PÉTROWSKI A., SIARRY P., TAILLARD E., _Métaheuristiques pour l'optimisation difficile_ , Eyrolles, 2003.

[GAN 98] GANLEY J.L., HEATH L.S., "An experimental evaluation of local search heuristics for graph partitioning", _Computing_ , vol. 60, no. 2, p. 121-132, 1998.

[GIL 06] GIL C., BAÑOS R., MONTOYA M.G., GOMEZ J., "Performance of simulated annealing, tabu search, and evolutionary algorithms for multi-objective network partitioning", _Algorithmic Operations Research_ , vol. 1, p. 55-64, 2006.

[HAN 01] HANSEN P., MLADENOVICÁ N., "Variable neighborhood search: principles and applications", _European Journal of Operational Research_ , vol. 130, p. 449-467, 2001.

[HOO 04] HOOS H.H., STÜTZLE T., _Stochastic Local Search_ , Morgan Kaufmann/Elsevier, 2004.

[ING 89] INGBER L., "Very fast simulated re-annealing", _Mathematical Computer Modeling_ , vol. 12, no. 8, p. 967-973, 1989.

[JOH 89] JOHNSON D.S., ARAGON C.R., MCGEOCH L.A., SCHEVON C., "Optimization by simulated annealing: an experimental evaluation, part I, graph partitioning", _Operations Research Society of America_ , vol. 37, no. 6, p. 865-892, 1989.

[KAR 98] KARYPIS G., KUMAR V., "A fast and high quality multilevel scheme for partitioning irregular graphs", _SIAM Journal of Scientific Computing_ , vol. 20, no. 1, p. 359-392, 1998.

[KIR 83] KIRKPATRICK S., GELATT C.D., VECCHI M.P., "Optimization by simulated annealing", _Science_ , vol. 220, no. 4598, p. 671-680, 1983.

[LAG 94] LAGUNA M., FEO T.A., ELROD H.C., "A greedy randomized adaptive search procedure for the two-partition problem", _Operations Research_ , vol. 42, no. 4, p. 677-687, 1994.

[LOR 03] LORENCO H.R., MARTIN O., STÜTZLE T., "Iterated local search", in Glover F., Kochenberger G. (eds), _Handbook of Metaheuristics_ , Kluwer, 2003.

[LOU 07] LOUREIRO R.Z., AMARAL A.R., "An efficient approach for large scale graph partitioning", _Journal of Combinatorial Optimization_ , vol. 13, no. 4, p. 289-320, 2007.

[MAR 96] MARTIN O.C., OTTO S.W., "Combining simulated annealing with local search heuristics", _Annals of Operations Research_ , vol. 63, p. 57-75, 1996.

[MET 53] METROPOLIS N., ROSENBLUTH A.W., ROSENBLUTH M.N., TELLER A.H., TELLER E., "Equations of state calculations by fast computing machines", _Journal of Chemical Physics_ , vol. 21, no. 6, p. 1087-1092, 1953.

[MLA 97] MLADENOVICÁ N., HANSEN P., "Variable neighborhood search", _Computers in Operations Research_ , vol. 24, p. 1097-1100, 1997.

[RES 03] RESENDE M.G.C., RIBEIRO C.C., "Greedy randomized adaptive search procedures", in GLOVER F., KOCHENBERGER G. (eds), _Handbook of Metaheuristics_ , Kluwer, Boston, 2003.

[ROL 96] ROLLAND E., PIRKUL H., GLOVER F., "Tabu search for graph partitioning", _Annals of Operations Research_ , vol. 62, no. 2, p. 209-232, 1996.

[SCH 99] SCHREIBER G., MARTIN O., "Cut size statistics of graph bisection heuristics", _SIAM Journal on Optimization_ , vol. 10, no. 1, p. 231-251, 1999.

[SIA 89] SIARRY P., DREYFUS G., _La méthode du recuit simulé: théorie et applications_ , ESPCIIDSET, Paris, 1989.

[SIM 97] SIMON H.D., TENG _S_.-H., "How good is recursive bisection?", _SIAM Journal on Scientific Computing_ , vol. 18, no. 5, p. 1436-1445, 1997.

[TAL 09] TALBI E.-G., _Metaheuristics_ , John Wiley & Sons, Hoboken, New Jersey, 2009.

[WAL 01a] WALSHAW C., A multilevel approach to the graph coloring problem, Technical Report 01/IM/69, School of Computing and Mathematical Science, University of Greenwich, London, United Kingdom, May 2001.

[WAL 01b] WALSHAW C., CROSS M., "A multilevel Lin-Kernighan-Helsgaun algorithm for the traveling salesman problem", _Future Generation Computer Systems_ , vol. 17, no. 5, p. 601-623, 2001.

[WAL 02] WALSHAW C., "A multilevel approach to the traveling salesman problem", _Operations Research_ , vol. 50, no. 5, p. 862-877, 2002.

[WAL 04] WALSHAW C., "Multilevel refinement for combinatorial optimisation problems", _Annals of Operations Research_ , vol. 131, p. 325-372, 2004.

[WAL 08] WALSHAW C., "Multilevel refinement for combinatorial optimisation: boosting metaheuristic performance", in Blum C., Aguilera M.J.B., Roli A., _S_ ampels M. (eds), _Hybrid Metaheuristics_ , Studies in Computational Intelligence, Springer, 2008.

[WIL 91] WILLIAMS R.D., "Performance of dynamic load balancing algorithms for unstructured mesh calculations", _Concurrency_ , vol. 3, p. 457-481, 1991.

1 Chapter written by Charles-Edmond BICHOT.

1 The Internet Website of the book, Graph partitioning: optimization and applications, found at the Web address: perso.ec-lyon.fr/charles-edmond.bichot/livre_partitionnement.

2 The Internet Website _Graph Partitioning Archive_ currently forms the largest database of test benches for graph partitioning in the world. It is shown in more detail in Appendix A.3, page 351.

# Chapter 7

# Population-based Metaheuristics, Fusion-Fission and Graph Partitioning Optimization 1

Following the previous chapter, Chapter 6, this chapter continues with the overview of metaheuristics adaptions for graph partitioning optimization problems. Whereas the previous chapter was dedicated to the adaptation of local search metaheuristics, and primarily focused on simulated annealing and iterated local search, this chapter describes the adaptation of population based metaheuristics and introduces the fusionfission technique.

This chapter starts with an introduction of ant colony algorithms, and their adaptation to graph partitioning optimization problems in section 7.1. Section 7.2 studies the application of evolutionary algorithms, and in particular genetic algorithms, to graph partitioning problems. Section 7.3 introduced a recent metaheuristic dedicated to graph partitioning, namely the fusion-fission method, which aims at reinforcing the multilevel process (introduced in Chapter 2) during the partitioning process.

# 7.1. Ant colony algorithms

Ant colony algorithms form a family of bio-inspired metaheuristics. These algorithms are inspired from the behavior of ants when they are looking for food, in order to solve optimization problems. The idea of using ant colony behavior to solve a combinatorial optimization problem is accredited to A. Colorni, M. Dorigo and V. Maniezzo [COL 92]. For more information about these algorithms, see [DOR 03, DOR 05, MON 10].

Ant colony algorithms are known for their ability to find good solutions to instances of the traveling salesman's problem1. Even though the traveling salesman's problem is a graph-based problem, like the partitioning problem, few ant colony algorithms have been proposed to solve the graph partitioning optimization problem. However, the nature of these two problems greatly differs: the traveling salesman's problem is to find a path, whereas the graph partitioning problem is to find sets of vertices.

A large number of ant colony algorithms have been grouped under the term ACO, which means Ant Colony Optimization. ACO algorithms construct solutions in the form of parameters associated with the components of a graph, by using pheromone trails of ants. ACO algorithms only imitate a single ant colony.

Unlike this mono-colony imitation, most of the graph partitioning ant colony algorithms deal with cases where several ant colonies struggle against each other. In this approach, several ant colonies are competing for food and each colony represents a part of the partition. Therefore, there are as many colonies as the number of parts in the partition.

P. Kuntz and D. Snyers were the first to initiate works on graph partitioning multiant colonies algorithms [KUN 94, KUN 97]. Their work triggered off a series of works, which were all based on the same approach. The ants cover a grid that represents their world. Ant colonies represent parts of the partition. Each box of the grid is associated with a vertex of the graph to partition. The association rule between a vertex and a box is often random. There is food in the grid. The amount of food in a box is proportional to the difference of cut by adding the box's vertex to the ant's part. Depending on the weight, one or several ants can – or cannot – transport this vertex to their nest. These works were extended in [LAN 99a, LAN 99b]. Subsequently, this approach has been combined with the multilevel method in [ROB 04, KOR 04]. In these latter works, an ant colony algorithm is used at each stage of the multilevel algorithm. All these works seem to provide good results related to the quality of the partitions found. However, the computation time is extremely longas compared with the multilevel method alone.

It was only recently that an ACO algorithm was used to solve the graph partitioning optimization problem. In [LEN 08, LEN 07], an ACO algorithm is used in conjunction with the multilevel method to minimize graph bisection cut. The initial results appear interesting. However the number of parameters to set, namely 8, is high and the computation time of 2 hours, still remains very expensive.

Contrary to the trend of the previous works, F. Comellas and E. Sapena proposed a single-colony ACO algorithm, without multilevel hybridization [COM 06]. Even though it is not a hybrid method, it can find very efficient partitions for small graphs (from 23 to 4,000 vertices).

Another multi-colony approach, different from the previous ones, was tested. This algorithm leads the ants straight to the graph being partitioned. There is therefore no projection of the graph onto a grid. Furthermore, it is no longer the vertices which contain the food, but the edges of the graph. This approach was proposed in order to solve the k-partitioning minimization cut problem with a flexible partitioning constraint [BIC 05, BIC 07b].

# 7.2. Evolutionary algorithms

Evolutionary algorithms form a well-known family of metaheuristics. Each metaheuristic of this family is inspired by Charles Darwin's theory of natural selection. These metaheuristics manipulate solutions, called chromosomes, grouped into a set that is called a population. The natural selection theory, which is applied to the population, is based on the three following principles:

– **variation** : chromosomes are different from each other;

– **adaptation** : chromosomes that are better adapted to their environment live longer and procreate more;

– **inheritance** : chromosomes characteristics are hereditary.

The principle of natural selection can be summarized as follows. Among the population, some of the chromosomes are better adapted to their environment than others. These chromosomes procreate more and their advantageous characteristics are inherited by the succeeding generations. Over a period of time, these characteristics become dominant within the population. This population is optimized with respect to its environment, i.e. objective function and constraints of the problem.

Evolutionary algorithms have emerged at the end of the 1950s [FRA 57, HOL 62, FOG 66]. Originally, three approaches of evolutionary algorithms had independently emerged: evolutionary strategies, evolutionary programming and genetic algorithms. Among these, genetic algorithms are certainly the better-known, mainly due to the reference works of D. Goldberg [GOL 89]. Genetic algorithms comply the generic scheme of an evolutionary algorithm, but with one special feature: during the evaluation stages, a solution of the problem is entirely made up from the genotype of a unique chromosome (this is often a chain of binary symbols).

## 7.2.1. _Genetic algorithms_

Genetic algorithms (GAs) were designed to solve optimization problems by using the principle of natural selection. Thus, some of the vocabulary from biological sciences has been used to describe this method. A chromosome is made of hereditary material, which means genes, where the genes characterize it and differentiate it from other chromosomes. Genes are elementary blocks, which characterize a solution. The objective function evaluates the adaptation of a chromosome to its environment. It is therefore also called the fitness function, noted _f_ () in this chapter. Genetic algorithms aim to maximize the fitness function. Thus, as for the cut graph partitioning optimization function, in the case of a single positive objective function to minimize, the inverse of this function can be used as the fitness function.

### 7.2.1.1. _Algorithm_

Algorithm 7.1 shows the generic stages of a genetic algorithm. In such an algorithm, the population evolves iteratively. The main stages of this evolution are the following:

1) an initial population of _nb_chromosomes_ is created. All the future generations are derived from this population. It is therefore better that this initial population covers a large part of the solution space of the problem; this is why it is often created randomly;

2) population evolves into successive generation through an iterative loop, with respect to three principles:

– a chromosome **adaptation** corresponds to the fitness value of this chromosome relative to the others;

– chromosomes **selections** form the population in a two stage process. First the future parents are selected, meaning the chromosomes which will be reproduced. This does not mean that, in this first stage, the other chromosomes disappear, since this is the subject of the second stage. This first stage is carried out by the selection operator. In the second stage, the chromosomes, which will form the new population, are selected. This selection is carried out by the replacement operator;

* * *

**Algorithm 7.1.** Genetic algorithm

* * *

1: **procedure** GA( _initial_population, nb_crossovers, nb_mutations_ )

2: _population ← initial_population_

3: _nb_chromosomes ← |population|_

4: **repeat**

5: _parents_ ← selection of _population_ 's chromosomes for reproduction

6: _of f spring_ ← ∅

7: **for** _i_ = 1 to _nb_crossovers_ **do**

8: child ← crossover of several chromosomes from parents

9: _of f srping ← {child} ∪ of f spring_

10: **end for**

11: **for** _i_ = 1 to _nb_mutations_ **do**

12: _child_ ← mutation of a chromosome of _parents_ or _of f spring_

13: _of f spring_ ← { _child_ } ∪ _of f spring_

14: **end for**

15: _population_ ← selection of _nb_chromosomes of of f spring ∪ population_

16: **until** stop criterion

17: **return** best adapted chromosome of _population_ (best fitness)

18: **end procedure**

* * *

– the **diversification** of the population allows the exploration of the solution space. The selected parents have three possibilities to generate children: crossover, mutation and a combination of these. The created chromosomes will be the new offsprings;

3) a stop criterion ends the search. It can either depend on the number of generations, the best chromosome's fitness or even time elapsed.

### 7.2.1.2. _Diversification operators_

Diversification allows the algorithm to explore the solution space searching for the global optima. The diversification stage consists of creating new solutions (i.e. chromosomes), the offspring, selected during the previous stage to form the set of parents, from the solutions present at the beginning of the iteration, the population. The designers of genetic algorithms were inspired by the living reproductive methods to create diversification operators. Crossover plays the role of sexual reproduction and mutation, those of a sexual reproduction:

– the **crossover** operator requires several parents, in general two, in order to produce several offsprings, in general as many as there are parents. This operator combines the genes (elementary blocks characterizing a solution) of the parents to create offsprings. The underlying idea is that it will be possible to replace several poor genes of a parent with the corresponding rich genes of another parent. This operator is therefore not necessarily random;

– the **mutation** operator consists of locally modifying one or several elements of the population. It is a very simple operator, but essential to the operation of evolutionary algorithms, since it guarantees diversification of a population. In certain issues, only this operator is used [FOG 66]. A simple way of creating a mutation is to randomly modify one or several genes of a chromosome, respecting the constraints of the problem.

The mutation operator is more often used to explore the solution space of the problem, and the crossover operator in order to reach the local optima. Therefore, when the mutation is sufficiently random, the crossovers can be more deterministic. The mutation and crossover operators are difficult to set. If the mutation operations are too simple or are not adequate, then mutation operations will close the algorithm in real time. On the other hand, if the operations are too many or two complex, then the operations will prevent the convergence of the algorithm. In addition, depending on the convergence scheme of the algorithm, an adapted diversification may be useful. For dealing with such difficult problems, adaptive operators can be used [EIB 99, YAN 02].

### 7.2.1.3. _Chromosomes selection_

Generally, the capacity of a chromosome to be selected, either for reproduction or replacement, depends on its adaptation, i.e. its fitness. The selection operator, like the replacement operator, sets the selection probability of each chromosome, in relation to its fitness and the fitness of other chromosones.

The best adapted chromosomes reproduce more often and replace the least well adapted. Therefore, copies of the best chromosome will invade the population after a certain time, called the domination time. The shorter this time, the higher will be the selective pressure. In such a case, there is a grave risk of premature convergence toward a local optimum. In fact, a super chromosome will invade the population, and the exploration of the search space will therefore become local, and will be centered on this chromosome. In the opposite case, i.e if the selective pressure is too low, the algorithm will not converge. The chromosomes will spread out in the solution space, without progressing.

The selective pressure, being very important in the case of genetic algorithms, is mainly detecrmined by the selection and replacement operators.

### 7.2.1.4. _Selection operators_

The aim of a selection operator is to choose the future parents in the population. Several selection operators exist:

– **tournament selection** : a uniform probability distribution is used to select chromosome couples from the population. In each selected couple, both chromosomes fight against each other. During combats, the better adapted chromosome wins with a probability between [0.5, 1]. This probability is a parameter of the method that will allow us to set the selective pressure. In this type of selection, a super chromosome will be selected several times and will occupy the place of several parents;

– **fitness proportional selection** (or roulette-wheel selection): this is a proportional method. Its name has emerged comes from an analogy of the random selection of casino roulette wheels. To each chromosome i a probability of being chosen is proportional to its fitness, _f_ ( _i_ ), as follows: _p i_ =  . Then, _nb_parents_ future parents are randomly selected according to a uniform distribution. The variance of this process is high. In the worst case, it is possible that no well-adapted chromosome is selected. This phenomenon, called _selection bias_ , can be limited with a large population. A variant called _stochastic universal sampling_ [DRÉ 03] also allows us to reduce this bias;

– **stochastic remainder without replacement** : this operator associates proportional selection and deterministic selection. A minimal number of representatives of each chromosome among the future parents is determined in advance, depending on the adaptation of the chromosome, then the population is completed through random selections:

\- the minimum number of representatives of a chromosome i is given by the integer part:   ,

\- the parents set is completed due to the roulette wheel selection, where the adaptation value of each chromosome is replaced by: _f_ ( _i_ ) ← _f_ ( _i_ ) −  .

### 7.2.1.5. _Replacement operators_

Replacement operator will ensure the proliferation of the best chromosomes, to the detriment of the worst. During this process, the survival probability of each chromosome must, therefore, correspond to its adaptation. Several replacement operators exist:

– **generational replacement** consists of keeping only the offspring in the next generation. In this case, the number of children will be equal to the number of chromosomes. Original genetic algorithms use this replacement;

– **evolutionary strategy replacement** is a deterministic selection of the nb_chromosomes best chromosomes among the offspring or among the parents and the offspring;

– **elitism** consists of keeping at least one chromosome possessing the best adaptation. This operator is coupled with another operator which selects the rest of the population. As pointed out in [JON 93], elitism can greatly improve the performance of a genetic algorithm, just as it can be detrimental, since it can lead to a too high selective pressure.

It is necessary to distinguish the selection operator from the replacement. The selection operator is assigned the task select the parents from the population, whereas the replacement operator determines the new population from the offspring and the parents.

## 7.2.2. _Standard process of genetic algorithm adaptation to graph partitioning_

Numerous genetic algorithm adaptations have been proposed to solve graph partitioning optimization problems. This section does not aim to introduce a stateof- the-art GA's adaptation, since it will be introduced in section 7.2.3 with the algorithm of Bui and Moon. However, it aims to describe several classical approaches to adapt a genetic algorithm to graph partitioning.

### 7.2.2.1. _Fitness function_

All the graph partitioning problems studied in this book are minimization problems. Since the different graph partitioning objective functions are all positive, it is therefore possible to consider the inverse of the objective function as the GA's fitness function for maximization.

### 7.2.2.2. _Population encoding_

As explained in section 7.2.1, a GA makes a population of chromosomes evolve, where each chromosomes corresponds to a solution of the problem. The main stages of a GA at each iteration are:

– parents selection;

– crossover;

– mutation;

– population replacement.

The encoding of the chromosomes must be explained for an in-depth understanding of the manner in which the operators work. In the case of an adaptation to graph partitioning optimization, the encoding of a chromosome corresponds to the encoding of a partition. More precisely, a chromosome is coded as an array of parts number (from 1 to _k_ , the number of parts of the partition), by proceeding going from vertex number 1 to vertex number _n_ ( _n_ being the number of vertices). Each element of this table contains its corresponding vertex part's number. Therefore, the size of a chromosome is equal to the number of vertices of the graph and, in the case of graph bisection, a chromosome has a binary representation (for example, for a graph with 6 vertices, a chromosome can be: 0 0 1 1 0 1).

### 7.2.2.3. _Crossover operator for graph partitioning_

In the case of graph bisection, since each chromosome has a binary representation, we can apply the following standard crossover operators:

– single point crossover [HOL 92];

– n-point crossover (multi-point);

– uniform crossover [ACK 87].

All these crossover operators require exactly two parents, _p_ 1 and _p_ 2, which produce two offsprings. However, only one offspring is usually kept. This offspring is either randomly chosen or chosen because of its better fitness.

The single point operator is presented in Figure 7.1. This operator begins by randomly selecting a cut point (a vertex). This cut point divides each parent into two blocks, which are in white and gray in the figure. Parents genes are copied into two children by merging, for each child, the opposite blocks of each parent into the new child. This type of operator is used in lot of GA's adaptation to graph partitioning optimization [TAL 91, MAI 94, MAN 96, ALA 99, APE 02]. The single point crossover operator can easily be generalized, so that it can become an _n_ -point operator. Therefore, instead of separating the chromosomes into two blocks, the _n_ -point operator separates them into n + 1 blocks. Of course, each of the randomly selected cut points is identical for both parents.

A simple way for describing the uniform crossover operator is to present it as a _n_ -point crossover where _n_ is random. This operator is illustrated in Figure 7.2 in the case where _n_ = 2. Let's describe this operator in more detail. A binary word, _m_ , with length the number of vertices of the graph, is randomly generated. When a value of the word _m_ is 0, the corresponding vertex in parent _p_ 1 is copied into child _of f_ 1

**Figure 7.1**. _Single point crossover opeator_

**Figure 7.2.** _Uniform crossover operator_

(respectively from parent _p_ 2 to child _of f_ 2). In the opposite case, when a value of _m_ is 1, the corresponding vertex in _p_ 2 is copied into _of f_ 1 (respectively from _p_ 1 to _of f_ 2).

All the previously proposed crossover operators tend to generate unbalanced bisections. Thus, they must be balanced in order to respect balance constraints. After being balanced, the offspring with the highest fitness is returned by the crossover. It should be noticed that the balancing process can greatly decrease the quality of the offspring. It should also be noticed that using these operators for direct k-partitioning should produce fairly random solutions, especially when _k_ is large. Another type of crossover operator can be considered in the case of _k_ -partitioning. This operator compares the parts against each other in order to determine the degree of similarity between each couple of parts. Since two parts containing the same vertices in two different partitions can be numbered differently, establishing this degree of similarity can take some time. Once the degrees of similarity are calculated, we effectively know the areas of similarity between the parents and we can proceed to a crossover which does not break them. In effect, these areas can contains dense subsets it would be detrimental to break. An operator of this type would be a multi-point crossover operator, but where all the points are not chosen randomly.

### 7.2.2.4. _Mutation operator for graph partitioning_

Two well-known GA's mutation operators can be easily adapted to graph partitioning: _deterministic mutation_ and _bit-flip_ mutation. Deterministic mutation consists of randomly changing parts number of a fixed number of randomly chosen vertices, whereas bit-flip mutation is applied to all vertices. Therefore, in bit-flip mutation each vertex will have a certain probability of changing of part of the partition. This probability must be very low to avoid getting in the case of a random search.

## 7.2.3. _The GA's adaptation to graph bisection optimization of BUI and MOON_

The work of T.N. Bui and B.R. Moon, [BUI 96], which relate to the adaptation of a GA to the graph partitioning optimization problem, is very often cited as an example of a metaheuristic adaptation to a graph partitioning problem. The graph partitioning problem studied in [BUI 96] is that of graph bisection when minimizing the cut and finding a perfectly balanced bisection (meaning that the parts of the partition are of equal sizes). The aim of this section is to present this adaptation that the authors have termed as _Genetic Bisection Algorithm_ (GBA).

The proposed adaptation differs from a direct application of GA by two important additions: offspring refinement through a local search method and the use of a preprocessing stage, before starting the algorithm. These additional steps allow to take into account the complexity of the partitioning problem, which is proportional to the size of the graphs. We will analyze it later.

A _memetic algorithm_ is defined as a GA that uses the same local search method to refine each chromosome of the population. Even though the authors do not mention it, it seems obvious that the GBA algorithm is a memetic algorithm.

Algorithm 7.2 shows the GA's adaptation of Bui and Moon to graph bisection optimization. The GBA carries out only one crossover and one mutation per iteration. As a result, it decreases the number of parameters of the GA by 2, and therefore simplifies its setup. The GBA proceeds as follows.

The preprocessing step modifies the numbering of the vertices of the initial graph. Then, the new numbered graph, _G″_ , acts as the reference graph for the rest of the algorithm. An initial population of _nb_pop_ chromosomes is randomly created. Each iteration of the algorithm begins by selecting two parents. From these two parents, the crossover operator only generates an unique offspring. A mutation is applied to this offspring. It is balanced afterward. Once the bisection balance is achieved, it is refined. The replacement depends on the quality of the produced offspring. If its cut is at least as good as its parents, then the offspring replaces its most related parent. The most related parent is the one who has the lowest Hamming distance to the offspring (it is called the _d_ function in Algorithm 7.2). Otherwise, the offspring replaces the least efficient chromosome of the population. The iterations continue until 80% of the chromosomes of the population have the same cut ( _note_ : however, two partitions can have the same cut and be extremely different). Finally, the GBA returns the chromosome with the lowest cut. The procedures used in this algorithm are described in more detail in the following sections.

### 7.2.3.1. _Parent selection_

To select two parents at each iteration, the GBA uses proportional selection technique, which is introduced in section 7.2.1.4. For this, the fitness function used is defined for every chromosome _i ∈ pop_ by:

* * *

**Algorithm 7.2.** The genetic bisection algorithm of BUI and MOON

* * *

1: **procedure** GBA(graph _G_ , number of parts k, population size nb_pop)

2: _G″_ ← preprocessing of _G_

3: _pop_ ← randomly creates _nb_pop k_ -partitions of _G″_

4: **repeat**

5: choose two parents, _p_ 1 and _p_ 2, in _pop_

6: _of f_ ← _crossover_ ( _p_ 1, _p_ 2)

7: _of f ← mutation_ ( _of f_ )

8: _of f ← balance_ ( _of f_ )

9: _of f ← refinement_ ( _of f_ )

10: **if** _cut_ ( _of f_ ) > _min_ ( _cut_ ( _p_ 1), _cut_ ( _p_ 2)) **then**

11: let _last ∈ pop_ such that _cut_ ( _last_ ) = max _i∈pop_ ( _cut_ ( _i_ ))

12: _last ← enf_

13: **else**

14: let _p_ ∈ { _p_ 1, _p_ 2} such that _d_ ( _of f, p_ ) = min ( _d_ ( _of f_ , _p_ 1), d(of f, _p_ 2))

15: _p ← of f_

16: **end if**

17: **until** 80% of _pop_ 's chromosomes have the same cut

18: let _p∗_ ∈ _pop_ such that _cut_ ( _p_ ) = _min i∈pop_( _cut_ ( _i_ ))

19: **return** _p∗_ converted for _G_

20: **end procedure**

* * *

where _cut_max = max j∈pop_( _cut_ ( _j_ )).

By using this fitness function, the probability that the most adapted chromosome is chosen is 4 times higher than that of the least adapted chromosome.

### 7.2.3.2. _Crossover, mutation and balancing_

The crossover operator of the GBA is shown in Figure 7.3. It begins by randomly dividing each parent into 6 blocks. It should be noted that the blocks are very different from the parts; the bisection of the parents into two parts is not used for such random separation into blocks. This operator creates two offsprings. However, the operator only returns the one with the lowest cut.

The first offspring, _of f_ 1, is a copy of the different blocks of _p_ 1 and _p_ 2, as shown in Figure 7.3. The first block is copied from same place refers _p_ 1 to the future parents in _of f_ 1 (i.e. the vertices are identical), the second block is copied from _p_ 2 to _of f_ 1, the third block is copied from _p_ 1 to _of f_ 1 and so on, alternately, up to the sixth block. _of f_ 1 is therefore created by using a 5-point operator (see section 7.2.2.3) The second offspring, _of f_ 2, is created slightly differently. For _of f_ 2, the blocks are copied in the

**Figure 7.3.** _Crossover operator of the GBA_

same order as _of f_ 1, starting with _p_ 1. However, if the blocks of _p_ 1 are copied as they are, each block of _p_ 2 copied in _of f_ 2 is the complementary of the original block.

Once the two offsprings are created, the mutation operator selects the most efficient and applies the mutation operator. The interest in the second offspring comes from the possible complementarity between two bisections. Indeed, if the encoding of two identical bisections is inverse, meaning if the part 0 of the first bisection corresponds to part 1 of the second bisection, This, in turn implies that the Hamming distance between themselves can be maximal. In order to avoid encountering this type of problem (two partitions are the same but with a different coding), the complementary of one of the parents is used. It can be noticed that this crossover operator is difficult to generalize to direct k-partitioning.

The mutation operator simply changes the part's number of some vertices of the offspring. A certain number of vertices is randomly selected and each corresponding vertex value in the offspring table is inversed (0 becomes 1 and vice versa) The number of vertices exchanged is randomly chosen between 0 and _n_ /100, where n is the number of vertices. This operator is a modified version of the bit-flip operator (this operator is described in section 7.2.2.4).

These mutation and crossover operators almost always create a very badly balanced offspring, since there is no mechanism to preserve the number of vertices in each part. Therefore, before continuing the GBA, a balancing stage is necessary. First, the number of vertices to be moved out of the part, _nb_dep_ , is calculated. Let's assume that for the bisection to be correctly balanced, the vertices should be moved from part 0 to part 1 (the idea is the same to balance from part 1 to part 0). In order to balance the bisection, a vertex _v ∈_ [1; _n_ ] is randomly chosen. Starting from this vertex v, the initial vertices in part 0 are moved to part 1. Then, the second vertex in part 0 is moved to part 1, and so on till _nb_dep_ vertices are moved from part 0 to part 1. If the _n_ -th vertex is reached before the end, the process continues with vertex number 1. This balancing process returns a balanced offspring _of f_ and also produces some mutation effects.

### 7.2.3.3. _Chromosome refinement_

It is known that GA are often more efficient for finding good overall solutions than local optima. The larger the size of chromosomes, farther they would be away from a local optimum. This problem is quite common with GAs, and to compensate for it,one usually uses local search methods to refine the chromosomes. In the case of GA's adaptation to graph partitioning, the size of the chromosomes is equal to the number of vertices of the graph and can therefore be very large. Thus, this problem of size may often affect the performance of the GA. In order to compensate and to have locally optimum chromosomes, the refinement of partitions found is systematic. In GBA, this refinement is carried out by a slightly modified version of the Kernighan- Lin algorithm. This algorithm is discussed in section 2.5.2.2, page 43. Of course, the refinement algorithm used in GBA uses the data structure proposed by Fiduccia- Mattheyses (this data structure is presented in section 2.5.3, page 47) which allows a linear time complexity.

Contrary to the original Kernighan-Lin algorithm, GBA refinement does not carry out several passes. This implies that, refinement is able to carry out only a single exchange of two sets of vertices, one for each part of the bisection. This permits a reduction in the overall GBA computation time. In addition, the maximum size of the exchanged vertices sets is also limited to _n_ /6 − 1, whereas it can go up to _n_ /2 − 1 vertices in the Kernighan-Lin algorithm.

### 7.2.3.4. _Preprocessing the graph_

The preprocessing step, proposed by Bui and Moon, involves applying a heuristic to the graph being partitioned for reorganizing its vertices. A group of strongly linked vertices can easily be found distributed in both parts of the bisection following the cutting into blocks carried out by the crossover operator. In fact, the numbering associated with these vertices being distributed between 1 and n, by dividing the bisection into 5 blocks these vertices will be appointed to different blocks. In order to reduce this problem, Bui and Moon propose a reorganization of the graph vertex numbering by using a breadth-first search algorithm (BFS), which moves from a randomly chosen vertex. Therefore, the vertices of the graph G will be reorganized depending on their respective distance. Finally, the preprocessing of the graph will create a graph _G″_ identical to the previous one, not counting the renumbering of its vertices.

### 7.2.3.5. _Evaluation_

The results presented in this section follows the same evaluation procedure as that of the ILS algorithms presented in Chapter 6, page 151. The results obtained by GBA for regular graphs are shown in Table 7.1. From these results, it can be noticed that

**Table 7.1.** _Comparing results of GBA, random searches using the GBA's Kernighan-Lin (RA KL) algorithm, and the two paragraph partitioning tools MeTiS and Scotch, on regular graphs_

**Table 7.2**. _Comparing results of GBA, random searches using the GBA's Kernighan-Lin (RA KL) algorithm, and the two paragraph partitioning tools MeTiS and Scotch, on ordinary/simple graphs_

despite the fact that the time for a GBA run is not limited, the algorithm does not converge. Therefore, the partitions found are quite mediocre. On the other hand, the partitions obtained in the case of ordinary graphs are of better quality, as presented in Table 7.2. However, GBA is several times slower than Scotch or MeTiS, with, in general, worse results.

The graphs used for this evaluation, whose name can be shown in Tables 7.1 and 7.2, can be downloaded from the Website of this book2.

## 7.2.4. _Multilevel evolutionary algorithm of Soper-Walshaw-Cross_

The adaptation of a genetic algorithm to graph partitioning optimization which has been proposed by Bui and Moon (presented in section 7.2.3) used a Kernighan-Lin's local optimization method for refining the partition. As shown in Chapter 2, this local optimization method is surpassed by the multilevel method. Therefore, in the same way as for the adaptation of simulated annealing or iterated local search (both presented in Chapter 6), it is quite obvious that we should improve the efficiency of the adaptation of genetic algorithms to graph partitioning through the use of a multilevel algorithm.

An original and efficient adaptation of a genetic algorithm using a multilevel algorithm was proposed by A. Soper, C. Walshaw and M. Cross in [SOP 00, SOP 04]. This adaptation, dubbed _J OSTLE Evolutionary_ (JE) by its authors, uses as multilevel algorithm a slightly modified version of the JOSTLE graph partitioning software. If the authors present their algorithm as an evolutionary algorithm, it is more precisely a genetic algorithm. The aim of this adaptation is to partition a graph into _k_ parts while minimizing the cut and respecting a maximum partitioning balance _bal_max_.

### 7.2.4.1. _Population encoding_

Contrary to the population encoding of classical adaptations previously introduced in section 7.2.2.2, JE uses as chromosomes variants of edges weighting of the initial graph G. More precisely, a bias table is associated with each vertex of the graph. A vertex's bias value represents the tendency that a cut may be adjacent to this vertex, i.e. one or more of its adjacent edges may be cut. Edges weights are calculated depending on the biases of the couple of vertices adjacent to them. Therefore, the crossover and mutation operators aim at modifying these bias tables. Contrary to the vertex weight, which is an integer, the bias associated with a vertex is a real value.

During a JE computation, chromosomes quality evaluation is carried out by the graph partitioning software JOSTLE. For this, the software has been modified to handle floating numbers, instead of integers in the algorithms. One of the main difficulties of this modification resides in the adaptation of the gain tables from the Fiduccia- Mattheyses method which are indexed by integers. In fact, one box of the table corresponds to a gain (which has an integer value). The actual values of the gains will be discretized, so that we can continue using the gain table. Let's take _e ∈ N_ , a real number _x ∈_ [ _e_ − 0, 5; _e_ \+ 05] will be ranked in the gain table in the _e_ gain box.

By using these modifications, JOSTLE could find partitions with the modified weighting of _G_ and thus evaluate the quality of these weightings. Even though the JOSTLE algorithm is stochastic, the use of a same seed for its random generator allows us to always find the same graph partition. Therefore, the evaluation of a chromosome of the population is costly, even if a tool such as JOSTLE is quick. If we add to this the fact that metaheuristics generally have a slower computation time, it is easy to understand, before even presenting the GA operators used, that this algorithm will be very slow. Its computation time is typically several days, but can reach several weeks [WAL 04].

The underlying idea of this method is to guide the choice of the JOSTLE partitioning algorithm. For this, a positive bias is added to the vertices of the graph, depending on their position in the partition found by JOSTLE. The bias of a vertex v is noted bias(v). The weight of an edge ( _v_ 1, _v_ 2) is equal to:

where _w_ ( _v_ 1, _v_ 2) is the weight of the edge in the original graph _G_. The partition's balance does not take into account the bias of the vertices, since this would affect its value. However, the application of a bias to the weight of the graph edges will have the effect of modifying the cut of the partition found by JOSTLE. The path of this cut will tend to consider those egdes whose weights have been reduced the least.

### 7.2.4.2. _Algorithm_

Algorithm 7.3 presents the JE adaptation of a hybrid multilevel GA to graph partitioning optimization. JE begins by creating 50 chromosomes each in a random way. More precisely, a bias table of size _n_ ( _n_ is the number of vertices of _G_ ) is created,

* * *

**Algorithm 7.3.** The _Jostle evolutionary_ algorithm of Soper-Walshaw-Cross

* * *

1: **procedure** JE(graph G)

2: _nb_chromosomes_ = 50, _nb_crossovers_ = 35, _nb_mutations_ = 15

3: _nb_iter_ ← 0

4: _population_ ← _create nb_chromosomes_ tables of random biases

5: **repeat**

6: _nb_iter ← nb_iter_ \+ 1

7: _of f_ spring ← ∅

8: for i = 1 to nb_crossovers do

9: _of f spring_ ← crossover of several chromosomes of population

10: _of f spring ← {of f spring} ∪ of f spring_

11: **end for**

12: **for** _i_ = 1 to _nb_mutations_ **do**

13: _of f srping_ ← mutation of a chromosome of population

14: _of f spring_ ← { _of f_ spring} ∪ _of f_ spring

15: **end for**

16: _population_ ← best _nb_chromosomes of of f spring ∪ population_

17: **until** _nb_iter_ ≥ 1,000

18: **return** best adapted chromosome of _population_

19: **end procedure**

* * *

for which each value is taken randomly in [0; 0.1]. The number of iterations of the GA is limited to 1,000. Even if this limitation does not seem excessive, for large graphs (several tens of thousands of vertices), the computation time of JE is several weeks [WAL 04]. At each iteration of the algorithm, each of the 50 chromosomes will generate the offspring of the following generation with the help of the crossover and mutation operators described in the following section. Once the offspring are created, the evolutionary strategy replacement operator select the new population of JE. This operator has been formerly described in section 7.2.1.5. The fitness function used is:

### 7.2.4.3. _Mutation and crossover operators_

In JE, the mutation and crossover operators modify the bias of the vertices. The parent selection strategy is simple: these operators randomly choose the parents in the population, but at the end of the mutation and crossover step, all chromosomes of the population must have been selected at least once. The following two paragraphs describe these operators.

The crossover operator chooses at least two parents to create an offspring. The vertices which are at the boundary of the cut of all the parents partitions are applied with a bias taken with a uniform probability of [0; 0.01]. A vertex at the boundary of the cut of a partition is a vertex that belongs to at least an edge cut by the partition found by JOSTLE. The other vertices are affected by a bias of value 0.1 added to the random number chosen in [0; 0, 01]. It appears that the authors have found these values empirically. Figure 7.4 illustrates this crossover operator in the case of two parents: Figure 7.4(a) shows parent no. 1 and Figure 7.4(b) shows parent no. 2. Figure 7.4(c) show the child graph. In this figure, the encircled vertices are the vertices that have the lowest bias. The two shaded vertices pointed by arrows are problematic. Indeed, these vertices can mislead the calculation of JOSTLE for obtaining the lowest partition's cut, 7.4(d). They represent the limit of this crossover operator.

As usual, the mutation operator only needs one parent. From this parent, i.e. a partition, it computes the set of vertices which have a distance less or equal to two vertices from an edge cut of this partition. That is to say, a vertex belongs to this neighborhood set, if it is either: a vertex belonging to an edge cut of the parent or a vertex of the direct neighborhood of the edge cut vertices. Then, to each vertex of this neighborhood set is associated a bias selected with a uniform probability in [0; 0, 01]. To the other vertices of the graph a bias of value 2.0 is applied, to which is added the previously selected bias.

### 7.2.4.4. _Evaluation_

According to the results published in [SOP 00, SOP 04] and those available in the _Graph Partitioning Archive_ (see the Appendix), during 2009-10, this algorithm was

**Figure 7.4.** _Illustration of the crossover operator of the J OSTLE evolutionary algorithm: partitions 7.4(a) and 7.4(b) are the two 4-partitions parents graphs; graph 7.4(c) is the child graph created by the crossover operator, its encircled vertices are those with the smallest bias; and finally, partition 7.4(d) represents the best possible partition (which may be found by JOSTLE in the algorithm)_

one of the most efficient for graph partitioning. At the end of 2008, around one third of the 816 best solutions listed on this Website were found with JE, which made it the most efficient algorithm, just ahead of _Iterated Multilevel Partitioning_ (this algorithm is described in Chapter 6, page 157). However, since it takes several days to several weeks of computation time to find these solutions [WAL 04], this algorithm is really not efficient, in terms of computation time. Thus, it seems that this adaptation is less efficient than the _Iterated Multilevel Partitioning_ algorithm which only takes several minutes of computation time to find similar quality partitions (unfortunately, here also, we do not have the exact durations).

## 7.2.5. _Other adaptations of evolutionary algorithms to graph partitioning optimization_

Numerous adaptations of evolutionary algorithms have been produced to solve the graph partitioning optimization problem. Without wishing to be exhaustive, this section lists some of these adaptations.

– **combination of multilevel and evolutionary algorithms**. The hybrid multilevelevolutionary method shown in [ALP 96] is one of the first hybrid approaches coupling a metaheuristic and a multilevel algorithm. As for example the _Jostle Evolutionary_ algorithm previously presented, other algorithms using a combination of these two methods have since been proposed [SOP 00, SOP 04, KÜC 05];

– **combination of spectral methods and evolutionary algorithms**. Spectral methods have also been hybridized with metaheuristics, and in particular genetic algorithms [MAR 05, MAR 06];

– **parallel evolutionary algorithms**. Numerous adaptations of parallel evolutionary algorithms have been proposed for the problem of graph partitioning optimization TAL 91, LAS 91, ALA 99, CHE 06, RAO 09b, RAO 09a]. For more information on the resolution of constraint graph partitioning optimization problem with a parallel approach, refer to [Chapter 4;

– **PROBE**. This is an acronym of Population Reinforced Optimization Based Exploration. It is a population based metaheuristic inspired from genetic algorithms and based on the study of similarities between chromosomes [BAR 04]. This new method is applied in [CHA 07] to perfectly balanced graph partitioning optimizing cut. It uses as search heuristic a region growing algorithm and as refinement method a Kernighan-Lin type algorithm;

– **memetic algorithms**. Memetic algorithms are genetic algorithms using a local search method to refine each chromosome of the population. The name of memetic algorithm is less widespread than the popular genetic algorithm, this is why certain authors do not use this terminology, like T.N Bui and B.R Moon (their algorithm is described in section 7.2.3). P. Merz and B. Freisleben present a memetic algorithm in [MER 98] which they compare to the GBA of T.N. Bui and B.R. Moon and simulated annealing of D. Johnson et al. (this algorithm is also introduced in Chapter 6, page 146). A memetic algorithm is proposed by S. Areibi and Y. Zeng in [ARE 04] for VLSI partitioning;

– **genetic algorithms**. Finally, numerous adaptations of genetic algorithms to the problem of graph partitioning optimization have been proposed in [MAI 94, HOH 96, MAN 96, BUI 96, GIL 98, APE 02, KAV 03, MAR 06].

One of the difficulties of the adaptation of evolutionary algorithms to the graph partitioning optimization problem resides in their slow convergence. Thus, these methods must be hybridized with efficient search algorithms in order to be applied to graphs with several thousand vertices. This problem came up during the evaluation of the GBA of T.N. Bui and B.R. Moon. However, as this section shows, there have been numerous studies carried out on the use of efficient search algorithms (local or global) by evolutionary algorithms such as the spectral or multilevel methods.

The graph partitioning optimization problem is difficult to solve through a directly parallel method. This problem is covered in Chapter 4 on the parallelization of graph partitioning. In addition, it can be emphasized that the placement of processes on parallel machines uses graph partitioning. Chapter 5 introduces this application. In this case, the use of evolutionary algorithms allows to propose parallel partitioning algorithms quite easily (as compared to a direct approach).

The efficiency of the adaptation of evolutionary algorithms in terms of the quality of partitions obtained has been proved by the _Jostle Evolutionary_ algorithm, but also by other algorithms, such as PROBE. However, their computation time still remains relatively long – one could even say too long – for obtaining a quality solution, when compared with multilevel algorithms used alone.

# 7.3. The fusion-fission method

Graph partitioning algorithms have to solve larger problems always. The graphs being partitioned can now include several million to several billion vertices (see Chapter 4 on parallelism and Chapter 14 on modularity). The multilevel approach, described in Chapter 2, which consists of coarsening the vertices, tends to lose its significance when the coarsened vertices are composed of tens or hundreds of thousands of vertices. In fact, these coarsened vertices themselves have such large graphs that they are very difficult to partition. In addition, since the refinement algorithm used inside the multilevel scheme is a local algorithm, its time and quality efficiency decreases when it has to handle such large data. It therefore, becomes necessary to use another method having a more global approach, in order to solve the partitioning problem of such large graphs. However, as we have seen in the previous sections, the metaheuristics using multilevel methods are not always able to adapt efficiently, both in terms of the quality of solutions found, and computation time. The fusion-fission method was created to be a global method allowing a fast and efficient partitioning of such graphs. For this, it oversees the multilevel method in its partitioning task.

This section introduces the fusion-fission method adapted to the problem of constrained graph partitioning3, that is to say minimizing the cut, under the constraint of a strong partitioning balance.

## 7.3.1. _Introduction_

Fusion-fission is a recent method for graph partitioning optimization BIC 07c, BIC 07b]. The fusion-fission method was originally designed to solve the problem of the partitioning of the European air space (this problem is described in Chapter 10). After having tested different classical methods for graph partitioning optimization, such as the multilevel method and several metaheuristics, it appears that this special air traffic control problem required a new method. In fact, this problem is an unusual problem of graph partitioning optimization, since the objective function is not the cut and the partitions can be strongly unbalanced. Thus, classical partitioning software such as Scotch or MeTiS do not give very good results. The first version of the fusion-fission algorithm designed to solve this air traffic control problem efficiently, is described in [Chapter 9, page 237.

**Figure 7.5.** _Place of the fusion-fission method among the multilevel method and refinement algorithms in the graph partitioning process_

Since then, this method has been generalized to other graph partitioning optimization problems, including the constrained one, i.e. minimizing the cut, under the constraint of an adjustable partition balance [BIC 08]. The idea of generalizing this method is to reinforce the effect of the multilevel algorithms in the partitioning process. For this purpose, the fusion-fission method guides the search carried out by the multilevel algorithm for finding more efficient solutions. Figure 7.5 shows the position of this new method in the partitioning scheme using the multilevel approach. Fusionfission managed graph subsets which will form the parts of the partition through union of some of these subsets, whereas the multilevel method coarsened vertices and the refinement algorithm remain restricted to the neighborhood of the vertices. Therefore, the fusion-fission method has a more global view of the problem to solve than the multilevel method, which itself has a more global view than a refinement algorithm4.

**Figure 7.6.** _Illustrated process of the fusion-fission method_

## 7.3.2. _Fusion-fission method principles_

The fusion-fission method makes a partition evolve during an iterative process, and more precisely it makes the parts of the partition evolve. To do so, it uses a multilevel algorithm as a local search method. We can recollect that the multilevel algorithm makes the coarsened vertices evolve during an iterative process. In order to get out of basins of attraction (energy sinks), fusion-fission uses the well-known mechanism of constraint relaxation of the problem. Therefore, it authorizes a partition to temporarily change its number of parts.

Figure 7.6 illustrates the main steps of an iteration of the fusion-fission algorithm. The algorithm starts from an initial partition _P k_ of the graph _G_ into _k_ parts. Each part of _P k_ is divided into _k′_ smaller parts (we will see later how _k′_ can be chosen). It is this division into _k′_ small parts which relaxes one of the constraints of the problem: the number of parts. This division into _k × k′_ small parts is enabled by a multilevel algorithm. Then, we obtain a partition _P_ _kk′_ of _G_ into _k × k′_ parts. Each part of this new partition _P_ _kk′_ becomes the vertex of a new graph _G″_ constructed from _G_. Subsequently, _G″_ is partitioned into _k″_ parts by the multilevel algorithm. _k″_ can be equal to k if we wish to intensify the search or to _k′_ (or another number of parts) if we prefer to diversify the search. The new partition _P′_ of _G″_ into _k″_ parts is projected onto _G_. A new partition _P k″_ of _G_ is therefore created. The last stage of the iteration involves refining _P k″_.

## 7.3.3. _Algorithm_

Algorithm 7.4 presents the adaptation of the fusion-fission method, when _k″_ = _k′_ , to the graph partitioning optimization problem. This algorithm is split into two blocks, an initialization part and an iterative loop, called _loop on the number of parts_ :

1. the fusion-fission algorithm initialization is a two step process:

1a. a sequence of integers is created which will give the number of parts of the new partition created at each iteration of the _loop on the number of parts_. In fact, as we have stated earliers, the number of parts of the partition found by the _loop on the number of parts_ changes at every iteration. Then, the number of parts of the new created partition must be known at the beginning of each of iteration. We chose to create the list of these numbers of parts at the beginning of the algorithm, even though other solutions are possible. The number of iterations of the _loop on the number of parts, n_ , is given as an argument of the algorithm. Thus, the algorithm will stop when this number of loops is reached. The creation of this number of parts sequence is the subject of section 7.3.5,

1b. an initial partition of _k_ parts is created by a graph partitioning tool, _partitioner_. Several choices are possible for selecting this partitioning method. However, in order to be fast and efficient, using a multilevel algorithm is a good choice. Moreover, a multilevel algorithm is used in the second block of the algorithm, the _loop on the number of parts_. Thus, it is easy to use the same multilevel algorithm for the creation of the initial partition, and this solution has been chosen in algorithm 7.4 More precisely, as we will see in section 7.3.4, the chosen multilevel algorithm is the pMetis graph partitioning tool. The partition of the initial graph into k parts found by the partitioning tool, partitioner, is stored in _P k_ . _P k_ also contains the most efficient partition found during the _loop on the number of parts_ , whereas _P_ is the current partition in this loop,

2. the _loop on the number of parts_ succeed to the initialization. This is a _n_ times loop, and each iteration is split into six steps;

2a. the first step of an iteration determines the number of parts that the new created partition should have. This number of parts is noted _k i_ at iteration _i_ and was determined

* * *

**Algorithm 7.4**. Adaptation of fusion-fission to graph partitioning optimization.

* * *

1: **procedure** FUSION-FISSION( _G_ = ( _V, E_ ), nb of parts _k, n, partitioner_ )

2: // Stage 1.a) of initialization

3: Create a random sequence of n integers according to the binomial distribution centered in _k_ : { _k_ 1, . . . , _k n_}

4: _k 0 ← k_

5: // Start stage 1.b)

6: _P k ← partitioner_( _V, k_ )

7: _P ← P k_ = { _P_ 1, . . . , _P k_}

8: // Stage 2.a) loop on the number of parts

9: **for** _k i = k_1 to _k n_ **do**

10: // Stage 2.b)

11: **for** _j_ = 1 **to** _k_ _i_ −1 **do**

12: Partition the part _P j ∈ P_ into _k i_ new parts with _partitioner_ ( _P j , ki_)

13: _parts_set_ ← addition of the _k i_ new parts created

14: **end for**

15: // Stage 2.c)

16: Create a graph _G′_ = ( _V′_ , _E′_ ) where each vertex is one of the parts of _parts_set_

17: // Stage 2.d)

18: _P_ = { _P_ 1,... , _P ki_} ← partitioner( _V′_ , _k i_)

19: // Stage 2.e)

20: _P′_ ← refinement of _P_

21: // Stage 2.f)

22: **if** _k i = k_ and _cut_ ( _P′_ ) < _cut_ ( _P k_ ) then

23: _P k ← P ′_

24: **end if**

25: **end for**

26: **return** _P k_

27: **end procedure**

* * *

by the number of parts sequence initialized in 1.a). As for the current partition, _P_ , it has _k_ _i_ −1 parts,

2b. this is the fission step. It consists of dividing each part of the current partition _P_ into several parts. Each part of the current partition _P_ , divided into _k_ _i_ −1 parts, is partitioned into ki new parts. Therefore, after this fission step, the partition _P_ will contain _k_ _i_ −1 ∗ _k i_ parts. Although another number of part other than ki may be chosen, during the fusion stage it will allow us to create more easily a new partition of ki parts, based on _P_ . It is the partitioning tool, partitioner, which partitions each of the _k_ _i_ −1 parts of _P_ into ki new parts,

2c. this step prepares the fusion step. A temporary graph, _G′_ , is created such that each vertex of this graph represents one of the _k_ _i_ −1 ∗ _k i_ parts of _P_ . The edges of this graph have for weight the sum of the weight of the edges linking the corresponding two parts in _P_ , and the vertices have for weight the sum of the weight of the vertices making up the corresponding part in _P_. At this stage, each vertex of _G′_ can be viewed as a big set of vertices of _G_ , or, with respect to the multilevel scheme as a set of coarsened vertices of _G_ ,

2d. this is the fusion step. It aims at creating a new partition of ki parts from the graph _G′_ constructed in the last step. This new partition, stored in _P_ , is created with the help of the partitioning tool, partitioner. Of course, this partition is created such that its cut is minimized whilst complying, approximately, with the partitioning balance. In fact, we noticed that with the refinement stage to follow, the cut was slightly more important than the balance at this stage. Once the partition into ki parts is created from the graph _G′_ , this partition is projected onto the initial graph _G_. This projection is trivial, since we know the mapping between vertices of _G′_ and vertices of _G_ ,

2e. this is the refinement step. During this step, the current partition, _P_ , is refined. The refinement replaces the nucleon emission stage of the atom present in the original fusion-fission model. The selection of a refinement tool is presented in section 7.3.6. After many tests, it appears that the refinement step is more useful for finding the best partition _P k_ in comparison to the ones being used as the current partition _P_ . In fact, not using the refined partition as current partition allows the algorithm to explore more local minimum than in the opposite case. This observation, at first seemingly non-intuitive, led us to use the unrefined partition to the detriment of the refined one during the next fusion-fission iteration,

2f. the last step of the iteration process update, if possible, the most efficient partition previously found, _P k_ . Of course, _P k_ is replaced by _P′_ only if | _P′_ | = k, if _cut_ ( _P′_ ) < cut(P k ), and if _P′_ strictly complies with the partitioning balance. However, if needed, the best partitions having a different number of parts than k can also be saved. This let us confirm the fact that good partitions into _k′_ parts would allow us to find good partitions into _k_ parts.

## 7.3.4. _Selection of the multilevel algorithm_

As there exist numerous implementations of the multilevel algorithm, several choices can be made. This choice should be made carefully because partitions quality and computation time of the fusion-fission algorithm greatly depends on it. In addition, the choice of this algorithm affects the balance of the partition to be found. Searching for a partitioning balance of, for example, 1.01, requires the use of an algorithm which can find a partition with such balance.

The graph partitioning tools _pMetis_ and _kMetis_ of the MeTiS library5 have been adapted to be used with the fusion-fission algorithm. The algorithm _pMetis_ produced partitions with a too unbalanced partitioning balance. Therefore, the results obtained with _pMetis_ were not good since the refinement algorithms were not able to sufficiently lower the partitioning balance of the partitions found by _pMetis_. However, the _kMetis_ algorithm worked very well. It allows to find efficient partitions in terms of cut, with a partitioning balance able to reach 1.03. However, reaching lowest partitioning balances is sometimes required; in that case, another multilevel partitioning algorithm should be chosen, or a balance partitioning algorithm should be used.

## 7.3.5. _Creation of the sequence of number of parts_

The variation of the number of parts of the partitions during the fusion-fission algorithm iterations is inspired from the variation of the number of atoms during the process of nuclear fusion and fission. In nuclear physics, when the number of nucleons is constant, then the variation of the number of atoms leads to a variation of the type of these atoms. The atom type, which maximizes binding energy between nucleons, is the iron atom. Thus, the variation of the number of atoms may leads, after some time, to a dominance of iron atoms. However, if the number of nucleons allows us to theoretically obtain _k_ iron atoms, a state possessing _k_ atoms can have no iron atoms. It is only successive reorganization of nucleons among themselves which allows us to obtain _k_ iron atoms.

The sequence of number of parts is inspired from the curve of average binding energy per nucleon shown in Figure 7.7. Locally, around the iron atom, the curve takes Binding energy between nucleons (MeV) Nucleons numbers of an atom

**Figure 7.7.** _Average binding energy per nucleon, as a function of the number of nucleons per atom_

the form of a small flattened hill, which means that the atoms near iron also have a strong binding energy. To imitate the binding energy curve and for simplicity reasons, in order to choose the number of parts, we used a binomial distribution centered on _k_ (the number of parts sought by the algorithm) and with a high variance in order to keep its flattened shape.

The binomial probability distribution,

has as expected value, _n bpb_ and as variance, _n bpb_(1 − _p b_).

In the context of the number of parts we are concerned with in our experiments, namely powers of 2 going from 2 to 64, we chose _n b_ = _k_ \+ 16 and  ,so that the curve is sufficiently flat and the number of parts are sufficiently different. The number _n b_ corresponds to the maximum number of parts that a partition can have during optimization.

The first step of the creation of the sequence of number of parts is to calculate, for each potential number of parts, its probability of occurrence. The potential numbers of parts are those between _n b_ − _k_ and _n b \+ k_. Hence, as the sum of the probabilities of a binomial distribution is equal to 1, to obtain n iterations of the _loop on the number of parts_ , each of these probabilities is multiplied by n. These numbers are then truncated to their lowest integer value. We therefore obtain the number of occurrences of each number of parts _k′_ ∈ { _n b_ − _k_ ,... , _n b_ \+ _k_ } in the loop:  . However, as these values have been truncated, their sum is strictly lower than n. The difference between their sum and n is added to the number of occurrences of the number of parts k. Therefore, each number of occurrences _no_ ( _k′_ ) of the number of parts _k′_ ∈ { _n b_ − _k_ ,..., _n b_ \+ _k_ } is equal to:

The second step consists of creating a sequence of number of parts _k′_ ∈ { _n b_ − _k_ ,... , _n b \+ k_} such that each of these _k′_ occurs exactly _no_ ( _k′_ ) times in the sequence. The sequence then undergoes a random permutation to mix the numbers of parts. This step concludes the construction of the sequence of number of parts.

## 7.3.6. _Selection of the refinement algorithm_

There are many graph partitioning refinement algorithms. Most of them are based on the Kernighan-Lin notion of gain. Several of these algorithms are described in Chapter 2, section 2.5 (page 40). Due to their number of choices, choosing one becomes quite complex. To make a relevant choice, we have established several selection criteria:

– we observed that the refinement part in the multilevel optimization process takes the largest computation time. Boosted by this experience, and in order to reduce the computation time of the fusion-fission algorithm, the refinement algorithm to be chosen must be quick;

– the refinement algorithm must be able to adapt itself to several different partitioning balances. In addition, if possible, the refinement algorithm may first balance the partition before refining it. In fact, some times, the partition to refine is not well balanced, thus a load distribution algorithm is sometimes useful;

– since the partitioning goal is to minimize the partition's cut, it is obvious that the refinement algorithm must be efficient.

The refinement step is the last step of the _loop on the number of parts_ before updating partition _P k_ 6. As we explained earlier, this step does not influence the current partition _P_ in the _loop on the number of parts,_ since the refined partition is only temporary stored in _P′_ , and then, in _P k_ if _P′_ 's cut is lower than _P k_ 's cut.

### 7.3.6.1. _Using the Global Kernighan-Lin refinement algorithm_

The three criteria presented above led us to choose a Kernighan-Lin type refinement algorithm by using the Fiduccia-Mattheyses implementation for more rapidity. One of the quickest refinement algorithms of this type is the Global Kernighan-Lin refinement (GKLR) algorithm. For the first time, we chose GKLR as refinement algorithm for the fusion-fission algorithm.

The usage of the GKLR algorithm inside fusion-fission was found to be very efficient when searching partitions with a low number of parts ( _k_ = 2 or 4). However, for higher numbers of parts ( _k_ = 16 or 32), the GKLR algorithm had experienced much difficulties while finding partitions which complied with the partitioning balance. Two reasons explain this. First, the GKLR algorithm was not designed to carry out load distribution. Second, the partitioning balance of the current partition _P_ in the _loop on the number of parts_ is very often higher for a large number of parts than for a small number of parts.

Therefore, due to the limitations of the GKLR algorithm describe above, it seems worthwhile to look for another refinement algorithm.

### 7.3.6.2. _Using the Walshaw-Cross refinement algorithm_

We therefothis section, is able to rere considered compensating for the inadequacies of the GKLR algorithm through the use of another refinement algorithm. The Walshaw-Cross refinement algorithm7, in addition to response to the three criteria presented at the beginning of turn a partition that can comply with the partitioning balance in all cases, even if it means that the partition cut should decrease. Hence the Walshaw- Cross refinement algorithm (section 2.5.6) uses a load distribution method, in order to balance the partition. In addition, its refinement algorithm is fast, since it is based on the implementation of Fiduccia-Mattheyses. Therefore, we have created a version of the fusion-fission algorithm, which uses the Walshaw-Cross refinement method in place of the GKLR algorithm, where the algorithm is compared to the fusion-fission/GKLR version in the next section.

## 7.3.7. _Evaluation_

The graphs used in this section, as well as links to the partitioning tools used, are available from the Internet site of the book8.

### 7.3.7.1. _Robustness of the fusion-fission algorithm_

This section presents the initial results of the test benches found by the fusion-fission algorithm. These results are published in [BIC 07a]. The fusion-fission algorithm uses a random sequence of _n_ integers, which follow a binomial distribution to create the _number of parts sequence._ In order to have a limited computation time, the number of iterations of the fusion-fission algorithm is restricted to _n_ = 1, 000.

The refinement algorithm used is the GKLR algorithm. The fusion-fission algorithm carries out four refinements at each iteration of its _loop on the number of parts_ , one for each value of the partition balance of the test benches: 1.00, 1.01, 1.02, 1.03 and 1.05. This adaptation of the algorithm is specially designed for benchmarking, since it does not require us to execute the fusion-fission algorithm for each of these partition balances. However, as the refinement step is the longest step of the fusionfission algorithm, repeating it four times at each iteration significantly lengthens the computation time of the algorithm.

The calculations shown in this section and those which follow have been carried out on a computer with a GNU/Linux Debian operating system, equipped with an Intel Premium IV processor rated at 3 GHz, having 512 Mb of RAM.

Figures 7.8 and 7.9 compare the cut found by the fusion-fission algorithm and those found by the JOSTLE and _pMetis_ graph partitioning tools9. The cut shown in these figures are, for each of the 19 graphs, the minimum, mean and maximum cut of 20 partitions found by the three algorithms. In order to simulate the seed change of the

**Figure 7.8**. _Minimal, mean and maximal cut of 20 bisections of 19 graphs with a partitioning balance of 1.01, respectively obtained by fusion-fission, J OSTLE and pMetis_

**Figure 7.9**. _Minimum, mean and maximum cut of 20 4-partitions of 19 graphs with a partitioning balance of 1.01, respectively obtained by fusion-fission, J OSTLE and pMetis_

random number generator function of the partitioning tools (the sources of JOSTLE are not available), the indices of the vertices of the graph have been permuted, without changing the graph structure. As the cuts of the partitions found are very different depending on the graph, the cuts of the partitions of each graph have been divided by the mean cut of pMetis. Thus, in Figures 7.8 and 7.9, a result is better the more it is below 0.

The fusion-fission algorithm is self-sufficient in the sense that it does not require any parameter other than the graph, the number of parts of the sought partition and the number of iterations of the algorithm (or its duration time).

The results found for partitions into 2 and 4 parts, as presented in Figures 7.8 and 7.9, show that the mean cuts of partitions found by the fusion-fission algorithm are lower than those of both software, and the differences between the maximum and minimum cuts are also much smaller than those of both software.

In view of the results presented in this section, the fusion-fission method seems clearly more robust10 than the JOSTLE and pMetis software.

### 7.3.7.2. _Evaluation on standard test benches_

Table 7.3 compares the cuts of the partitions found by the fusion-fission algorithm and those found by Scotch and MeTiS on different bench graphs. For the latter software, its two variants: recursive bisection (pMetis) and direct partitioning (kMetis) are used.

It should be noticed that the two MeTiS tools searched partitions for a small but unfixed partitioning balance. In fact, partitions found both by pMetis or kMetis can have a balance of 1.00, 1.01, 1.03 or more, depending on the graph and the number of parts. On the opposite side, the Scotch graph partitioning tool has a parameter for defining a strict balance adjustment.

Two different refinement algorithms are used with the fusion-fission algorithm: the Global Kernighan-Lin refinement algorithm (GKLR) presented as FF in Table 7.3 and the Walshaw-Cross refinement algorithm presented as FFbis.

The results shown for MeTiS and Scotch correspond to the minimum cuts found after 30 minutes of iterations of these algorithms. This implies that for testing the robustness of these tools, we used a simple iterative algorithm. This algorithm initially applied the tool to the graph, stored the result, and then permuted the graph, so as to

**Table 7.3.** _Comparison of the different versions of the fusion-fission algorithm for k = 2 and k = 4_

simulate the seed change, before repeating the iteration. It is this iterative algorithm which is limited to 30 minutes.

The results obtained by the two versions of the fusion-fission algorithm were obtained after 10 minutes of computation time, and at the most, 2 hours. It should be noted that 4 partitioning balances are searched at the same time in this algorithm, thus 2 hours of computation is roughly equivalent to 30 minutes when searching for only one partitioning balance as SCOTCH and MeTiS do.

The results presented in Table 7.3 show, through both its algorithm versions, that the fusion-fission method brings an increase in efficiency, as compared when a multilevel algorithm is used alone. However, even though the encoding of fusion-fission has not been optimized and the solutions are found for four simultaneous partitioning balances, its computation time is long.

# 7.4. Conclusion

In this chapter and the previous chapter, we have shown that simulated annealing, iterated local search, ant colony algorithms and genetic algorithms can be adapted to graph partitioning optimization. These different methods, part of the large family of metaheuristics, are generic optimization methods for the resolution of difficult problems. The great genericity of these methods makes them very adaptable to different problems and often allows several adaptations of these methods to the same problem.

It is very difficult to compare these methods with each other from a general point of view and present one as being better than the others. This assertion is known as the _no free lunch_ theorem. This comment remains valid when these methods are adapted to the different graph partitioning optimization problems. However, in the case of graph partitioning optimization, there is a method that gives better results than these different adaptations of metaheuristics, the multilevel method outperforms. In fact, as much in terms of the quality of the partitions found as for the computation time, the _multilevel method surpasses these adaptations_. Undoubtedly, this acknowledgment has led most of the authors when adapting metaheuristics to the graph partitioning optimization problem, to hybridize these with the multilevel method.

The most important observation which can be made from these two chapters is the omnipresence of methods hybridizing a metaheuristic with a multilevel method. Only this hybridization allows the different current adaptations of metaheuristics to find better quality partitions than the multilevel method. Even then, these solutions have a much longer computation time.

The future use of metaheuristics, in the case of graph partitioning optmization, seems therefore to be limited to hybridization with the multilevel method. However, this method can also be seen as a metaheuristic [WAL 04, WAL 02, WAL 01]. It would, however, seem that the favored domain of this method is graph partitioning optimization. We can therefore ask ourselves whether the multilevel method is the only metaheuristic type method designed for graph partitioning which enables such performances?

It was in this frame of mind that fusion-fission had evolved. That is to say, it was through searching for a metaheuristic-type method specific to graph partitioning, that fusion-fission was found. This method aims to explore an extra dimension, which has not been truly explored by the multilevel method, namely the organization of subparts between themselves. Indeed, the multilevel method focuses on the aggregation of vertices, where the aggregations reach their limits of representativeness in the case of large graphs. A new layer is imposed to better represent the initial structure of the graph: the subparts. It is the role of the fusion-fission method to optimize their arrangement in the partitioning.

# 7.5. Acknowledgments

This chapter and the previous one owe much to the numerous suggestions of Nicolas Durand and Patrick Siarry, as well as the comments of François Pellegrini, Jean-Baptiste Gotteland, Jean-Marc Alliot, David Gianazza and Pascal Brisset. I would like to thank them for the time they spent re-reading my work and for their help.

# 7.6. Bibliography

[ACK 87] ACKLEY D., _A Connectionist Machine for Genetic Hillclimbing_ , Kluwer, 1987.

[ALA 99] ALAOUI S.M., FRIEDER O., EL-GHAZAWI T., "A parallel genetic algorithm for task mapping on parallel machines", in _Proceedings of the IEEE IPDPS Workshop on Bio-Inspired Solutions to Parallel Processing Problems in Conjunction_ , 1999.

[ALP 96] ALPERT C.J., HAGEN L.W., KAHNG A.B., "A hybrid multilevel/genetic approach for circuit partitioning", in _Proceedings of the IEEE Asia Pacific Conference on Circuits and Systems_ , p. 298-301, 1996.

[APE 02] APETREI A., GHEORGHIES O., LUCHIAN H., DRECHSLER R., "An evolutionary approach to graph partitioning", _Evolutionary Methods for Design Optimisation and Control_ , 2002.

[ARE 04] AREIBI S., ZENG Y., "Effective memetic algorithms for VLSI design automation = genetic algorithms + local search + multi-level clustering", _Evolutionary Computation_ , vol. 12, no. 3, p. 327-353, 2004.

[BAR 04] BARAKE M., CHARDAIRE P., MCKEOWN G.P., "The probe metaheuristic and its application to the multiconstraint knapsack problem", _Applied Optimisation_ , vol. 86, p. 19-36, 2004.

[BIC 05] BICHOT C.-E., ALLIOT J.-M., "Optimisation par colonies de fourmis appliquée au découpage de l'espace aérien européen en zones de qualification", _RIVF_ , 2005.

[BIC 07a] BICHOT C.-E., "Application of fusion fission to the multi-way graph partitioning problem", in _Proceedings of the Combinatorial Tools for Parallel Sparse Matrix Computation Workshop, in conjunction with PPAM'07_ , Lecture Notes in Computer Science, p. 698-707, September 2007.

[BIC 07b] BICHOT C.-E., Élaboration d'une nouvelle métaheuristique pour le partitionnement de graphe : la méthode de fusion-fission. Application au découpage de l'espace aérien, PhD thesis, Institut National Polytechnique de Toulouse (INPT), November 2007.

[BIC 07c] BICHOT C.-E., "A new method, the fusion fission, for the relaxed k-way graph partitioning problem, and comparisons with some multilevel algorithms", _Journal of Mathematical Modeling and Algorithms (JMMA)_ , vol. 6, no. 3, p. 319-344, Springer, 2007.

[BIC 08] BICHOT C.-E., "A new meta-method for graph partitioning", in _Proceedings of the 2008 IEEE Congress on Evolutionary Computation_ , p. 3498-3505, June 2008.

[BUI 96] BUI T.N., MOON B.R., "Genetic algorithm and graph partitioning", _IEEE Transactions on Computers_ , vol. 45, no. 7, p. 841-855, 1996.

[CHA 07] CHARDAIRE P., BARAKE M., MCKEOWN G.P., "A PROBE based heuristic for Graph Partitioning", _IEEE Transaction on Computers_ , vol. 52, no. 12, p. 1707-1720, 2007.

[CHE 06] CHEVALIER C., PELLEGRINI F., "Improvement of the efficiency of genetic algorithms for scalable parallel graph partitioning in a multi-level framework", in _Proceedings of Euro- Par06_ , vol. 4128 of Lecture Notes in Computer Science, p. 243-252, 2006.

[COL 92] COLORNI A., DORIGO M., MANIEZZO V., "Distributed optimization by ant colonies", in _Proceedings of the 1st European Conference on Artificial Life_ , Elsevier Publishing, p. 134- 142, 1992.

[COM 06] COMELLAS F., SAPENA E., "A multiagent algorithm for graph partitioning", in _EvoWorkshops_ , Lecture Notes in Computer Science, p. 279-285, 2006.

[DOR 03] DORIGO M., STÜTZLE T., "The ant colony optimization metaheuristic: algorithms, applications, and advances", in Glover F., Kochenberger G. (eds), _Handbook of Metaheuristics_ , Kluwer, 2003.

[DOR 05] DORIGO M., BLUM C., "Ant colony optimization theory: a survey", _Theoretical Computer Science_ , vol. 344, no. 2-3, p. 243-278, 2005.

[DRÉ 03] DRÉO J., PÉTROWSKI A., SIARRY P., TAILLARD E., _Métaheuristiques pour l'optimisation difficile_ , Eyrolles, 2003.

[EIB 99] EIBEN A.E., HINTERDING R., MICHALEWICZ Z., "Parameter control in evolutionnary algorithms", _IEEE Transaction on Evolutionary Computation_ , vol. 3, no. 2, p. 124-141, 1999.

[FOG 66] FOGEL L.J., OWENS A.J., WALSH M.J., _Artificial Intelligence through Simulated Evolution_ , Wiley, 1966.

[FRA 57] FRASER A., "Simulation of genetic systems by automatic digital computers", _Australian Journal of Biological Sciences_ , vol. 10, p. 484-491, 1957.

[GIL 98] GIL C., ORTEGA J., DIAZ A., MONTOYA M., "Meta-heuristics for circuit partitioning in parallel test generation", in _Proceedings of the IEEE IPDPS Workshop on Biologically Inspired Solutions_ , 1998.

[GOL 89] GOLDBERG D., _Genetic Algorithms in Search, Optimization and Machine Learning_ , Addison-Welsey, 1989.

[HOH 96] HOHN C., REEVES C., "Graph partitioning using genetic algorithms", in _Proceedings of the 2nd International Conference on Massively Parallel Computing_ , p. 27-43, 1996.

[HOL 62] HOLLAND J., "Outline for a logical theory of adaptive systems", _Journal of the ACM_ , vol. 9, no. 3, p. 297-314, 1962.

[HOL 92] HOLLAND J.H., _Adaptation in Natural and Artificial Systems_ , MIT Press, 1992.

[JON 93] DE JONG K.A., SARMA J., "Generation gaps revisited", in Whitley L.D. (ed.), _Foundations of Genetic Algorithms_ 2, p. 19-28, Morgan Kaufmann, 1993.

[KAV 03] KAVEHA A., BONDARABADYB H.R., "A hybrid graph-genetic method for domain decomposition", Finite Elements in Analysis and Design, vol. 39, p. 1237-1247, 2003.

[KOR 04] KOROŠEC P., ŠILC J., ROBI  B., "Solving the mesh-partitioning problem with an ant-colony algorithm", _Parallel Computing_ , vol. 30, no. 5-6, p. 785-801, 2004.

[KÜC 05] KÜCÜKPETEK S., POLAT F., OGUZTÜZÜN H., "Multilevel graph partitioning: an evolutionary approach", _Journal of the Operational Research Society_ , vol. 56, no. 5, p. 549- 562, 2005.

[KUN 94] KUNTZ P., SNYERS D., "Emergent colonization and graph partitioning", in _3rd International Conference on Simulation of Adaptive Behavior: From Animals to Animats 3_ , p. 494-500, 1994.

[KUN 97] KUNTZ P., LAYZELL P., SNYERS D., "A colony of ant-like agents for partitioning in VLSI technology", in _Proceedings of the European Conference on Artificial Life_ , p. 417-424, 1997.

[LAN 99a] LANGHAM A.E., GRANT P.W., "A multilevel k-way partitioning algorithm for finite element meshes using competing ant colonies", in _Proceedings of the ACM Genetic and Evolutionary Computation Conference_ , vol. 2, p. 1602-1608, 1999.

[LAN 99b] LANGHAM A.E., GRANT P.W., "Using competing ant colonies to solve k-way partitioning problems with foraging and raiding strategies", in Proceedings of the 5th European Conference on Advances in Artificial Life, vol. 3172 of Lecture Notes in Computer Science, p. 621-625, 1999.

[LAS 91] VON LASZEWSKI G., "Intelligent structural operators for the k-way graph partitioning problem", in _Proceedings of the International Conference on Genetic Algorithms_ , p. 45-52, 1991.

[LEN 07] LENG M., YU S., "An effective multi-level algorithm based on ant colony optimization for bisecting graph", in _Advances in Knowledge Discovery and Data Mining_ , vol. 4426 of Lecture Notes in Computer Science, p. 138-149, 2007.

[LEN 08] LENG M., YU S., DING W., GUO Q., "An effective multi-level algorithm based on ant colony optimization for graph bipartitioning", _Journal of Shanghai University_ , vol. 12, no. 5, p. 426-432, 2008.

[MAI 94] MAINI H., MEHROTRA K., MOHAN C.K., RANKA S., "Genetic algorithms for graph partitioning and incremental graph partitioning", in _Proceedings of Supercomputing_ , p. 449- 457, 1994.

[MAN 96] MANIKAS T., CAIN J., Genetic algorithms vs. simulated annealing: a comparison of approaches for solving the circuit partitioning problem, Report no. TR-96-101, University of Pittsburgh, 1996.

[MAR 05] MARTIN J.G., "Subproblem optimization by gene correlation with singular value decomposition", in _Proceedings of the ACM Genetic and Evolutionary Computation Conference_ , p. 1507-1514, 2005.

[MAR 06] MARTIN J.G., "Spectral techniques for graph bisection in genetic algorithms", in _Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation_ , p. 1249-1256, 2006.

[MER 98] MERZ P., Freisleben B., "Memetic algorithms and the fitness landscape of the graph bi-partitioning problem", in _Parallel Problem Solving from Nature V_ , vol. 1498 of Lecture Notes in Computer Science, p. 765-774, 1998.

[MON 10] MONMARCHÉ N., GUINAND F., SIARRY P., _Artificial Ants_ , ISTE, London, John Wiley & sons, New York, 2010.

[RAO 09a] RAO A.R.M., "Distributed evolutionary multi-objective mesh-partitioning algorithm for parallel finite element computations", _Computers and Structures_ , vol. 87, p. 1461-1473, 2009.

[RAO 09b] RAO A.R.M., "Parallel mesh-partitioning algorithms for generating shape optimized partitions using evolutionary computing", _Advances in Engineering Software_ , vol. 40, p. 141- 157, 2009.

[ROB 04] ROBI  B., KOROŠEC P., ŠILC J., "Ant colonies and the mesh-partitioning problem", in Dorigo M., Stützle T. (eds), _Ant Colony Optimization_ , MIT Press, 2004.

[SOP 00a] SOPER A.J., WALSHAW C., CROSS M., A combined evolutionary search and multilevel optimization approach to graph partitioning, Report no. 00/IM/58, University of Greenwich, 2000.

[SOP 04] SOPER A.J., WALSHAW C., CROSS M., "A combined evolutionary search and multilevel optimization approach to graph-partitioning", _Journal of Global Optimization_ , vol. 29, p. 225-241, 2004.

[TAL 91] TALBI E.G., BESSIERE P., "A parallel genetic algorithm for the graph partitioning problem", in _Proceedings of the ACM International Conference on Supercomputing_ , p. 312- 320, 1991.

[WAL 01] WALSHAW C., CROSS M., "A multilevel lin-kernighan-helsgaun algorithm for the traveling salesman problem", _Future Generation Computer Systems_ , vol. 17, no. 5, p. 601- 623, 2001.

[WAL 02] WALSHAW C., "A multilevel approach to the traveling salesman problem", _Operations Research_ , vol. 50, no. 5, p. 862-877, 2002.

[WAL 04] WALSHAW C., "Multilevel refinement for combinatorial optimization problems", _Annals of Operations Research_ , vol. 131, p. 325-372, 2004.

[YAN 02] YANG S., "Adaptive non-uniform crossover based on statistics for genetic algorithms", in _Proceedings of the Genetic and Evolutionary Computation Conference,_ p. 650-657, 2002.

1 Chapter written by Charles-Edmond BICHOT.

1 The traveling salesman's problem consists in finding the shortest path which links a set of towns, knowing their distances between each other.

2 The Website of the book _Graph partitioning: Optimization and Applications_ , can be found at the Web address: perso.ec-lyon.fr/charles-edmond.bichot/.

3 The constrained graph partitioning optimization problem has been shown in Chapter 1, page 13.

4 Some refinement algorithms are described in Chapter 2, page 40, including that which is certainly the most well-known graph partitioning refinement algorithm, the Kernighan-Lin algorithm.

5 For more details about the MeTiS graph partitioning library, see the Appendix.

6 _P k_ is the current best partition found by the fusion-fission algorithm.

7 The Walshaw-Cross refinement algorithm is described in detail in Chapter 2, section 2.5.6, page 50.

8 The Website of the book, _Graph partitioning: optimization and applications_ , can be found at the Web address: perso.ec-lyon.fr/charles-edmond.bichot/.

9 More details about the JOSTLE and MeTiS graph partitioning tools are provided in theAppendix.

10 We define the robustness of a non-deterministic algorithm, by its capacity to find similar cost results with different random number generator seeds.

# Chapter 8

# Partitioning Mobile Networks into Tariff Zones 1

# 8.1. Introduction

The control of congestion in mobile networks depends on the spatial and temporal distribution of the traffic. When the congestion is pinpointed, the rating is an effective way of indicating the temporal redistribution of the demand. A planned rating model is proposed by calculating the new price grids based on the behavioral model of the customers. Hence, the cellular architecture of a mobile network involves taking into account the properties specific to different cells. Therefore, the study is confronted with the technical difficulties that are related to the diversity of the cell properties and with the cognitive constraints that are related to the perception of the pricing policy by the users. Zone rating therefore presents an effective solution to overcome these problems. The process of dividing the network into tariff zones, also called _space division_ is carried out, owing to the formalization of the problem, which is in the form of a graph partitioning optimization problem. This formalization is the main subject of this chapter.

## 8.1.1. _Scheduled rating model_

The mobile networks, like all transport networks, are subjected to congestion problems. In circuit mode, the mobile networks such as GSM [LAG 99] and congestion on voice communications result in the rejection of calls. Call rejection refers to the part of the demand, called _rejected traffic_ , which is not satisfied. The calls that are admitted to the network represent the satisfied demand, called _observed traffic flow_ , and the demand itself is sometimes called as _demand offered_ or _traffic offered_. This demand is measured in _Erlang_ [ERL 17].

When congestion is localized in time, the rating is an effective way of indicating the controlling demand. A new tariff policy will support the network usage during the off-peak periods and try to inhibit the usage during the peak hours. The new price grids are calculated by using a scheduled rating model [OUG 08]. This model was designed to use at an individual cell scale, and is generalized to a complete network.

### 8.1.1.1. _Rating model for a cell_

The control of congestion in the cell consists of reducing the demand during the busy periods, and increasing it during the off-peak periods. A better use of resources can be obtained by optimizing the temporal distribution of the demand. Consequently, the objective of the rating model is to redistribute the demand in the cell. The temporal redistribution of the demand which are initially rejected during peak hours tries to flow completely or partly in the traffic during the off-peak periods.

### 8.1.1.2. _Objective function of the optimization problem_

The problem of controlling the congestion for rating presents itself as an optimization problem aiming to minimize the traffic that is rejected in the cell. Its modeling is translated mathematically by the minimization of the square deviation between the demand of peak periods and the capacity of the cell _n_ :

[8.1]

where   is the capacity of the cell _n_ , _N Per_ is the number of periods in a day, and   is the estimated demand in the cell _n_ during the period _i_ after the application of the new price grid. _ρ_ ( _n, i_ ) _e _is calculated by using the following mathematical expected value:

[8.2]

where   is the initial demand in the cell n during the period _i_. _prob n,j_ ( _i_ ) is the probability whose calls are postponed from one period to another depending on the tariff changes. _prob n,j_ ( _i_ ) represents the level of traffic transferred from period _j_ to period _i_ , so the traffic taken out of _j_ to be flowed during the period of _i_. This amount depends on the behavior of clients and their sensibility to tariff fluctuations, its estimation requires the updating of a client behavior model.

### 8.1.1.3. _Client behavior model_

The objective of the client behavior model is to model the reaction of clients in the aim of predicting their behavior faced with price fluctuation. This model allows us to calculate the probability, _prob n,j_ ( _i_ ).

From a macroscopic point of view, _prob n,j_ ( _i_ ) is seen as the probability which will pass the demand from period _j_ to period _i_. It represents a measure of attractiveness of period _i_ compared to period _j_ , which is based on the behavior of clients, by expressing their reaction when they are faced with tariff fluctuation. This measurement allows an evaluation of the impact of a new price grid on their behavior.

From a microscopic point of view, _prob n,j_ ( _i_ ) is considered as the probability which means that the clients, who used the network initially during the period _j_ , are influenced by the fluctuation of prices and finally decided to change the call period in the favor of _i_. This is, therefore, a choice of a new call period _i_.

The problem of choice of the call period can be reduced to a known mathematical problem in literature under the name of discrete choice problem. The mathematical models which cover the discrete choice problems, called as probabilistic discrete choice models, are of many types [KEN 03]: the linear model, the _Probit_ model, the _Logit_ model, and so on.

The _LOGIT_ ( _Logistic Probability Unit_ ) model [BEN 03] is the most commonly used model in the field of transport; it allows us to calculate the probability by choosing an alternative among a set of alternatives. To calculate this probability of choosing a period from a set of periods, all that needs to be done is to adapt the general formula of the _Logit_ model in the following way:

[8.3]

where _U n,j_ ( _i_ ) is a utility function, which expresses the utility of _i_ in relation to _j_ , and αn is a strictly positive factor, which determines the influence of the utility function on _prob n,j_ ( _i_ ).

The function _U n,j_ ( _i_ ) describes the utility of transferring the call from the period _j_ to the period _i_. It is defined according to the two parameters1:

– a price parameter, which describes the sensibility of the clients to the applied price grid;

– a _preference_ parameter, which describes the sensibility and preference, objective or subjective, of the clients for a period.

The tests that were carried out show the efficiency of the scheduled rating model for a network cell. The objective is, therefore, to generalize the model to a complete network.

## 8.1.2. _Rating model for a network_

The results obtained by the scheduled rating model in a cell must be generalized to the scale of a complete network. The concept of a complete network designates a set of cells managed by a same operator. The objective of this section is to propose a generalization of the scheduled rating model to calculate a unique price grid for this set of cells.

### 8.1.2.1. _Adaptation of the model_

The generalization of the scheduled rating model, equation [8.1], to a complete network is conveyed by the global optimization of use of all the network cells. At this stage, this concerns in calculating a unique price grid, which minimizes the combined rejected traffic of all the cells, according to the following function:

[8.4]

This generalization sets aside the difficulties of implementation. The application of the model to a complete network is confronted with the difficulties and operational constraints of the operator, and cognitive constraints from the users.

The concept of network implies taking into account the specific properties related to the cellular architecture of a mobile network. Therefore, the generalization of the scheduled rating model is, in particular, confronted with the technical difficulties that are related to the diversity of cell properties and with the cognitive constraints that are related to the perception of the tariff policy.

### 8.1.2.2. _Technical difficulties and cognitive constraint_

From the operator's point of view, the extension of the scheduled rating model represents the implementation difficulties essentially to do with the characteristics of the traffic demand in the cells. The diversity of the demand profiles and non-homogeneous dimensioning of the cells are the two major difficulties encountered which we will explain in detail in the following section.

**Figure 8.1.** _Diversity of demand profiles (load per hour)_

In order to illustrate the problem of the demand profile diversity, let us consider the example of the two cells whose demand profiles are shown in Figure 8.1. The demand profiles are represented by the curves; the X -axis represents the 24 hours of a day and the Y -axis represents the offered traffic demand measured in Erlang [ERL 17]. The horizontal line represents the capacity of the transmitter in Erlang. In this example, cell 1 shows the congestion during the evening time between 7 pm and 11 pm; therefore, the new price grid calculated by the model will increase the price in the evening and decrease the price during the day time. In addition, for cell 2 congested during the daytime between 10 am and 4 pm, the model will increase the price during this period. It is clear that if the model processes these two cells simultaneously, it is not easy to calculate an adequate unique price grid for both the cells at the same time.

The problem of non-homogeneous dimensioning of cells is shown in Figure 8.2. Consequently, in the previous example, the congestion appeared in different periods, whereas the two cells have the same capacity. In this example, the congestion appears in the same periods for both cells. Hence, the problem concerns with the cell dimensioning. As a result, cell 1 has a capacity of 9 Erlang, whereas cell 2 has only a capacity of 2.9 Erlang. The processing of these two cells must take into account, for each cell, the ratio between the values of the demand during different hours.

**Figure 8.2.** _Non-homogeneous dimensioning of cells (load per hour)_

The aim of this work is to offer the mobile network operator, a tool allowing the establishment of a price grid to optimize the use of their network. This indicative price grid has the goal of influencing the behavior of clients. Its readability is, therefore, an important requirement which forms a cognitive constraint. The stability of a price grid is considered as a spatial constraint which was linked to the principle of zone rating.

In effect, the technical difficulties encountered during the generalization of the rating model brings out an idea of zone rating. A zone is a set of cells, geographically distributed, having similarities in its demand profiles. For cognitive reasons, notably the readability of the price grid, the contours of the zones must be clearly defined, that is to ensure the geographical connexity between the cells of a same zone. Figure 8.3 shows an example of network division into the rating zones. This example presents itself as being the ideal case where we can clearly distinguish the three zones which can be, respectively, the town center, the outskirts, and the open spaces..

### 8.1.2.3. _Resolution of the model extension problem_

Faced with the difficulties mentioned above, a potential solution organizes the network into groups of cells by using an automatic classification method [FOR 65]. The principle of the approach consists of using the _k-means_ algorithm [MAC 67, BAL 67] to

**Figure 8.3.** _Rating zones_

**Figure 8.4.** _Regrouping into two groups_

**Figure 8.5.** _Regrouping into three groups_

group the cells of the network according to the similarities of their demand profiles and then treat each group of cells as a sub-network. Therefore, it is possible to calculate, for each group considered as a zone, a unique price grid, hence the principle of zone rating.

Several tests have been carried out on a real example of 190 cells covering the two towns: one is a small-sized town and another is of medium-sized town. The cells have different demand profiles with capacities differing from one cell to another. Figure 8.4 shows an example of results obtained with a regrouping into two groups of cells and Figure 8.5 shows a regrouping into three groups. This graphical representation of the network cells in a map is obtained by the Voronoï diagram deduced from Delaunay triangulation [FOR 92]. The cells of the same group are shown by the same color.

The efficiency of the cell regrouping has been evaluated by using the correlation analysis2 of the cell groups demand profiles for each regrouping. The results of this analysis show that the regrouping allows us to obtain the groups of cell which show strong similarities.

Even though the regrouping method of the cells with the _k-means_ algorithm satisfies the similarity criterion of the demand profiles, it has however, been proven to be insufficient to determine the geographical contours of the rating zones, that is to take into account the geographical connexity of the cells of the same zone, and therefore to meet the cognitive constraint.

As a result, other than the similarities in the demand profiles, the cells of the same zone must show the geographical connexity expressed by the existence of a path between each pair of cells of the zone. Therefore, it is sufficient to consider the cells as being vertices for the problem to be modeled by a graph. As a result, the problem of spatial division is considered as a graph partitioning optimization problem.

For the rest of the chapter, we mean the spatial division of the network as the process of dividing the network into geographically distributed rating zones. The cells of each zone respect the following two criteria: they show demand profile similarities and geographical connexity. The spatial partitioning, therefore, consists of clearly defining the contours of the rating zones.

# 8.2. Spatial division of the network

The integration of the geographical parameter into the construction of the rating zones has led to a new formalization of the network space division problem. This formalization considers that the network is a complete graph where the cells are vertices of the graph, and each pair of cells ( _n, m_ ) is linked by an edge weighted by both weights, which represent, respectively, the similarities of the demand profiles and the geographical proximity of the cells. It is, therefore, about solving a complete graph partitioning optimization problem. The weights of the edges are mathematically formalized based on the definitions stated below..

## 8.2.1. _Definitions_

### 8.2.1.1. _Appearance of a demand profile_

Let us consider ρ as a demand profile. We call the _appearance of ρ_ the curve, which represents its overall aspect (Figure 8.6).

### 8.2.1.2. _Pearson correlation_

The Pearson correlation [WEL 00] or _Pearson product-moment correlation coefficient_ is a value, situated between −1 and 1, which measures the strength of

**Figure 8.6.** _Appearance of a demand profile (load per hour)_

similarity between the two vectors (equation [8.5]). It is called as Pearson coefficient and is written as _Corr_ ( _ρ_ 1, _ρ_ 2):

[8.5]

where   and _σ_ ( _ρ_ ) are, respectively, the mean and the standard deviation of the vector _ρ_.

The Pearson coefficient indicates the strength of the linear dependence between the two vectors, which increases as it approaches an absolute value of 1. If we have _Corr_ ( _ρ_ 1, _ρ_ 2) = ±1, _ρ_ 1 and _ρ_ 2 are linked by a relationship _ρ_ 1 = _a_ ∗ _ρ_ 2 \+ _b_. The value 1 indicates that the two vectors are perfectly correlated. When this value equals −1, the vectors are opposed, they are _anti-correlated_. A correlation value of 0 simply means that the vectors are not at all correlated.

### 8.2.1.3. _Similarity of cell demand profiles_

The exact meaning which we attribute to the term similarity strongly influences its mathematical modeling. In the dictionary, there exist several definitions. In the LAROUSSE dictionary (French dictionary), for example, similarity literally means more or less perfect resemblance. In geometry, the term is used to express a property that the two figures possess a similar shape. In the context of space division, the meaning which we attribute to this term answers both the definitions at the same time, it designates the resemblance of the demand profiles graphical curves..

Let us take two demand profiles,   and  , where   is the value of the demand in the cell _n_ during hour _i_. In terms of similarity between the two profiles _ρ_ 1 and _ρ_ 2, the resemblance of the appearance of the curves resulting from the graphical representation of the two vectors _ρ_ 1 and _ρ_ 2.

Figure 8.7 illustrates an example of the demand profile similarity. The profiles _ρ_ 1 and _ρ_ 2 show significant similarities, whereas the profile ρ3 shows very low similarities compared to the profiles _ρ_ 1 and _ρ_ 2. The similarity can therefore be more or less perfect, average, low or even mediocre. As said, it is possible to quantify the resemblance between the two demand profiles _ρ_ 1 and _ρ_ 2 by calculating a correlation coefficient of the two vectors _ρ_ 1 and _ρ_ 2, respectively. We use the Pearson correlation to measure this correlation.

Therefore, the similarities between the cells of a network are given by a matrix Ψ, called as _similarity matrix_ :

[8.6]

The following matrix compares the correlations of the cell demand profiles shown in the example of Figure 8.7:

This matrix shows that the Pearson correlation efficiently measures the similarities of the demand profiles. In effect, the value of the correlation between the profiles 1 and 2, which are well correlated, is very high (0.84), whereas the profiles 1 and 3, which are much less correlated, have a low value (0.17). The profiles 2 and 3, which have a negative value, are placed between the non-correlated and anti-correlated cases.

### 8.2.1.4. _Geographical proximity of the cells_

The notion of cell proximity is related to the geographical neighborhood. Two cells are said to be from the _first neighborhood_ , if the geographical areas covered by their respective antennas are connected. For example, in Figure 8.9, the cell 8 is a neighbor of the cell 12.

A cell _n_ is said to be inth neighborhood of the cell, _m_ if _n_ and m are separated by ( _i_ −1) other cells. In the example of Figure 8.8, cell 3 belongs to the first neighborhood

**Figure 8.7.** _Example of demand profile similarities per hour_

**Figure 8.8.** _Example of geographical neighborhood of the cells_

of cell 16, cell 22 belongs to its second neighborhood, cell 15 to its third neighborhood, and so on.

The mathematical formalization of the neighborhood constraints is given by a square matrix, _V_ , called as _neighborhood matrix_. An element _V n,m_ of _V_ is called as _neighborhood distance_ :

∀ _n_ , _m, Vn,m_ = _i_ if _n_ belongs to the _i_ nth neighborhood _m_.

## 8.2.2. _Formalization of the space division problem_

The space division problem considers the network as a graph whose vertices are the cells and the edges are the connections to be optimized between each pair of cells. The objective is to maximize both the similarity and connexity of the cells in each of the parts, without setting the constraints on the number and the distribution of the cells within the parts. As a result, the problem is considered as non-constrained and the choice of the adequate objective function is strongly linked to this characteristic. This choice makes up one of the crucial stages in the resolution of a graph partitioning optimization problem.

### 8.2.2.1. _Objective function for partitioning_

The objective functions which we look to optimize in a non-constrained graph partitioning problem vary. The cut ratio [WEI 89] and the standardized cut cost [SHI 00] are the two functions which most often occur in the literature, with improvements or even very close variants.

The calculation formula of the cut ratio, strictly related to the weight _W k_ and the cut   of each part _k_ of a partition, has been adapted to the network space division problem. This adaptation has two original aspects:

– on one hand, it allows taking into account the geographic dispersion of the cells through the calculation of the standard deviation of neighborhood distances;

– on the other hand, it allows us to avoid the dominance of zones having a few number of cells, by integrating the number of edges into the calculation formula of the weight and the cut of a zone.

Therefore, the objective of the space division is to minimize the global ratio cut/weight of the partition that is the ratio of each of the zones. The cost function is, therefore, given by the following equation:

[8.7]

where _N Zone_ is the number of zones that we wish to construct.

### 8.2.2.2. _Weight and cut of a zone_

The weight of a zone _S k_ , written as _W k_ , is a measurement which quantifies the relationship existing between the cells of this zone. It aims to maximize the similarities of the profiles and minimize the neighborhood distances, as shown in the following equation:

[8.8]

where _σ_ _n,m_ ∈ _S k ;m_> _n_ ( _V n,m_) is the standard deviation of the neighborhood distances of the zone cells and _η k_ is the total number of relationships between the cells of the zone _S k_ , that is the number of edges of the part _S k_ of the graph. _σ_ _n,m_ ∈ _S k ;m_> _n_ ( _V n,m_) allows us to take into account the dispersion of the values and favor the best geographically regrouped cells. _η k_ allows us to put into perspective the weight of the zone in relation to the number of cells it contains. As a result, the sum of the neighborhood distances in a zone is high as large as the number of cells. Therefore, the zones having a minimum number of cells always have a strong weight and are therefore favored. Yet, these zones are not always the best in terms of geographical neighborhood. Figure 8.9 shows the effect of these two parameters on the weight calculation of the zones.

Let us take an example of a network, where we are interested in the 10 cells numbered in Figure 8.9. The three examples called as _a_ partitioning, _b_ partitioning,

**Figure 8.9.** _Examples of partitioning_

and _c_ partitioning show, respectively, a zone comprises of the cells: {1, 2, 3, 4, 5}, {1, 2, 3, 4, 10}, and {1, 2, 3, 4, 5, 6, 7} (gray tinted cells). The following matrix _V_ represents the neighborhood distances of the 10 cells:

By admitting the similarities of the cell demand profiles which are perfect, that is ∀ _n, m_ : Ψ _n,m_ = 1, let us study the influence of the standard deviation on the weight calculation of the zone. Table 8.1 shows the values of the weights obtained with and without standard deviation, and then by adding the number of relationships between the cells.

**Table 8.1.** _Effect of standard deviation and the number of arcs on the zone weight calculation_

**Table 8.2.** _Example of cost function for two zones_

| Zone 1 | Zone 2
---|---|---
Weight
Cut
Cost function | −0.8
0.5
−0.62 | 0.7
0.5
0.56

Table 8.1 shows that the weight   calculated without taking into account the standard deviation and the number of relationships between the cells of the partitioning zone _a_ , which is higher than _c_. This means that a is better than _b_ , which is better than _c_. Yet, in reality, the cells of _c_ are geographically grouped better than that of _b_. The introduction of the standard deviation into the formula corrects in part this limit by showing that _c_ is better than _b_. However, a still remains better than _c_ , whereas _c_ must be better in the sense if it is preferable, than the zones having a maximum cell count. The introduction of the number of relationships between the cells of the zone allows us to obtain the results conforming with the objectives.

If the weight of a zone quantifies the relationship between the cells and the zone itself, the cut of a zone,  , quantifies the relationship between each cell of this zone and the cells of other zones:

[8.9]

where   is the number of relationships between each cell of the zone _k_ and the cells of the other zones.

### 8.2.2.3. _Normalization of the similarity matrix_

The similarity values are the measurements which we use in the calculation of the cost function during the optimization process. These values can either be positive or negative. Thus, the sign of the cost function value of a solution is crucial for the results. Let us consider an example given by Table 8.2 which shows the calculation of the cost function for the two zones.

In Table 8.2, zone 1 has a lower cost value than zone 2. If we are looking to minimize the objective function, zone 1 is, therefore, better. Yet, this zone has a much more mediocre weight than zone 1.

To overcome this problem, we propose to standardize the correlation matrix of the cells by shifting the definition domain from [−1,1] to [0,1]:

[8.10]

**Table 8.3.** _Standardization of the correlation matrix_

| Zone 1 | Zone 2
---|---|---
Weight
Cut
Cost function | 0.1
0.5
5 | 0.95
0.5
0.53

Both the zones in the previous example will, therefore, have cost functions as shown in Table 8.3. The latter shows that the standardization of the values in the correlation matrix of the cells allows to calculate the correct cost functions. The formality, therefore, defined allows us to solve the problem of the space division now through the heuristic method which is shown in the following section.

## 8.2.3. _Resolution of the space division problem by a genetic algorithm_

In the previous section, the space division was modeled in the form of a non-constrained graph partitioning problem. In the literature, there exist numerous methods to solve this type of problem (see Chapters 6 and , which cover the adaptation of metaheuristics to the graph partitioning optimization problem). Among these methods, we find metaheuristics originated in the 1980s to solve the difficult optimization problems [DRÉ 03]. Their stochastic nature allows them to explore a very large solution space, without going through the calculation of the gradient. For the space division problem, the choice rests on a resolution by a genetic algorithm [FRA 57, HOL 62] because of the simplicity in adapting this type of algorithm to the presented problem, and their capacity to process very large-sized problems.

This section presents the genetic algorithm used for the optimization of rating zone contours.

### 8.2.3.1. _The genetic algorithm_

For the construction of rating zones, a genetic algorithm is implemented in order to optimize the zone contours. Algorithm 8.1 shows this genetic algorithm. The different phases of this algorithm are described below.

### 8.2.3.2. _Encoding and chromosome evaluation_

A chromosome represents a possible solution to this problem, as it contains a set of informations necessary for the evaluation of the objective function ( _fitness_ ). The encoding of the chromosome must therefore be complete, if it allows the representation of all possible solutions, as it is obvious that if a solution is not represented, the algorithm can never reach it.

* * *

**Algorithm 8.1.** Genetic algorithm for space partitioning

* * *

1: **procedure** Space Partitioning Genetic Algol

2: _Pop_ ← random generation of the initial population

3: **repeat**

4: Evaluation of the current population

5: _N ewGeneration_ ← _N bElites_ best individuals of _Pop_

6: **while** _Size_ ( _Pop_ ) < _NbPop_ **do**

7: _Parents_ ← select two parents from _Pop_

8: _Of f spring_ ← cross two parents

9: Mutate the offspring

10: _NewGeneration_ ← _Of f spring_

11: **end while**

12: _Pop_ ← _NewGeneration_

13: _NewGeneration_ ← ∅

14: **until** Stop Criterion

15: **end procedure**

* * *

**Figure 8.10.** _Example of chromosome encoding with zone number of the first line and cell number on the second line_

Each solution of the problem processed must contain two important pieces of information:

– the numbers of the rating zones;

– the cell numbers of each zone.

The representation of the solutions is, therefore, given by real encoding of the chromosome which associates, to each cell, the number of the rating zone to which it belongs. Figure 8.10 shows an example of a chromosome which conveys a solution, with three zones, and are described as follows: the first is composed of the cells 2, 4, 5, 9, 11, and 12 (in dark gray), the second is composed of the cells 1, 3, 8, and 10 (in white), and the third is composed of the cells 6 and 7 (in light gray).

The size of all the chromosomes is equal to the number of sites of the considered example of the mobile network. Each gene corresponds to a value between 1 and the maximum tolerated number of zones λ to construct, knowing that the resolution algorithm may lead to better results with a number of zones lesser than the λ.

The evaluation of a chromosome is associated with a qualitative value. The latter is directly related to the quality of the zones in terms of profile similarity and geographical neighborhood. The construction of the contours of the rating zones helps in searching for a partition of the network, which minimizes the sum of the cut/weight ratios of the partition zones. This objective is given by the equation [8.7], which allows us to evaluate each chromosome.

### 8.2.3.3. _Generation of individuals of the initial population_

The initial population is created randomly. During the generation of numbers of the rating zones, only the constraint of the maximum number of zones is to be respected. As a result, the value of a gene is taken according to a uniform distribution on the interval [1, λ], λ representing the maximum number of zones. A random generation of individuals, therefore, allows a uniform distribution of the solutions through the search space; and therefore, the diversification of the initial population shows better results.

### 8.2.3.4. _Parent selection_

An important stage participating in the evolution of the population is parent selection, which considerably influences the result of the algorithm. The selection strategy chosen for our problem combines the roulette-wheel mechanism with elitism.

The principle of proportional selection (or roulette-wheel) is specified in Chapter 7, page 168. The proportional selection favors the selection of the best evaluated individuals.

The principle of genetic algorithms is to evolve the generations successively in order to obtain an offspring of better quality. The roulette-wheel selection mechanism which we introduced allows us to give stronger reproductive chances to better suited individuals. However, it is possible to lose the best individuals over generations since, in genetic algorithms, two parents of good quality do not necessarily produce a good offspring. In order to guarantee the survival of the best individuals, while implicating the reproductive process, the selection process is completed by an elitism mechanism. It consists of keeping the _N bElites_ best individuals::

### 8.2.3.5. _Crossover and mutation operators_

There exist several general crossover operators in the literature which have proved their efficiencies. However, for certain problems, a specific operator will prove to be the best. The algorithm is, therefore, tested with two crossover operators: a two-point crossover (multipoint crossover is introduced in Chapter 7, page 170), and a specific crossover called as directed multipoint crossover.

The two-point crossover offers the possibility of leaving the free algorithm in the exploration of the search space, thereby increasing the changes in converging toward a global optimum. Nevertheless, this convergence risks being very slow.

However, in order to avoid a possible loss in the chromosome structure, mainly due to the brutality of the two-point crossover operator which sets aside the relationships built between the cells, the choice of crossover points is not random. It follows a rule which consists of choosing a percentage _P_ of the worst genes as crossover points. This crossover is called as _directed multipoint crossover_ (see Algorithm 8.2).

* * *

**Algorithm 8.2.** Directed multipoint crossover operator

* * *

1: **procedure** Directed Multipoint Crossover

2: **for all** _parent_ ∈ _Parent_ s **do**

3: CrossGene ← {P % worst genes}

4: **for all** _i_ ∈ _CrossGene_ **do**

5: Permute the gene _i_ of both parents

6: **end for**

7: Add the new chromosomes in _Of f spring_

8: **end for**

9: **end procedure**

* * *

The evaluation of genes is obtained by associating, to each gene _i_ , a value   which measures its assignment quality to its zone _z_.   evaluates the weight of the cell _i_ in relation to the other cells of its zone _z_ :

[8.11]

The genetic algorithm is used with two mutation operators: a random mutation of a gene and a mutation of the worst genes.

In effect, the objective of the mutation is to improve a solution by bringing out a slight modification in one or several genes. The mutation method proposed, called as _worst gene mutation_ (see Algorithm 8.3), consists of selecting the gene having the lowest assignment quality (equation [8.11]) and modifying its value _i_ for each zone. The new value is randomly selected according to a uniform distribution.

* * *

**Algorithm 8.3.** Controlled mutation operator

1: **procedure** Worst genes mutation

2: **for** _z_ = 1   _NZone_ **do**

3: **for** _i_ = 1   _NCell_ **do**

4: **if**   = min( _q z_) **then**

5: Random selection of a number _a_ ∈ {1, 2, . . . , _N Zone_ }

6: **if** _a_ ≠ _z_ **then**

7: The value of the gene _i_ is replaced by _z_

8: **end if**

9: **end if**

10: **end for**

11: **end for**

12: **end procedure**

* * *

# 8.3. Experimental results

The algorithm has been tested with different setups. The example shown here is the one which gives the best result:

– population size: 1,500 individuals;

– number of elite individuals: 10 individuals;

– percentage of bad genes for directed crossover: 30%;

– crossover rate: 100%;

– mutation rate: 100%;

– stop criterion: the algorithm stops if the solution is not improved after 50 generations.

The tests are organized into two batches: the first batch is carried out by using the crossover operators and random mutations, and the second batch is carried out by using the specific operators which was described previously.

Table 8.4 shows that the algorithm finds the best solutions in terms of cost function when we use the specific operators. This is explained by the fact that these operators act on the bad genes, and are therefore more efficient at reducing the conflicts over generations. This result is confirmed by the computation time. Thanks to a shorter computation time, the random operators have come across with much larger number of solutions, without compromising in terms of the quality of the final result. The faster computation is due to the fact that the specific operators carry out an evaluation of all the genes during each genetic operation.

The results of Table 8.4 show that, even though the crossover and mutation operators offer a better convergence time of the algorithm, the specific operators are

**Table 8.4.** _Comparison of algorithms_

**Figure 8.11.** _Division into two zones_

**Figure 8.12.** _Division into three zones_

more efficient in reaching the better quality solutions after having covered a smaller area of the search space.

For each series of tests, the algorithm has to run several times by varying the number of zones each time. Figures 8.11-8.14 show the results obtained for divisions into 2, 3, 4, and 5 zones, respectively. The example of network used for these tests contains 190 cells covering two towns, small and medium size, respectively.

The division into two zones illustrated in Figure 8.11 shows each town and its outskirts as a single zone.

When we carry out the division into three zones (Figure 8.12), a center part of the medium-sized town is distinguished as a new zone.

The transition into four zones (Figure 8.13) makes the center of the small town appear as a zone. These results show that the algorithm measures the importance of the similarities of the demand profiles. The analysis of these profiles shows that the

**Figure 8.13.** _Division into four zones_

**Figure 8.14.** _Division into five zones_

difference in similarity between the profiles of the town center and those of the outskirts is more significant for the medium-sized town.

Finally, the division into five zones (Figure 8.14) shows that the center cells of the medium-sized town are reorganized and produce two distinct zones.

These results indicate that the formalization of the space division problem compared to the graph partitioning optimization problem is effective. The objective function defined and the optimization algorithm used were able to carry out a spatial partitioning of the network into rating zones. However, this partitioning method is not perfect and if necessary adjustments can be made if necessary depending on the case.

# 8.4. Conclusion

The application domains of the graph partitioning tools are several and varied. This chapter has introduced an example of application in the field of mobile networks. The cell structure of this type of network facilitates the modeling in the form of graphs by considering the cells as vertices. As a result, the division of the network into rating zones is instinctively considered as a graph partitioning optimization problem. By choosing the adequate objective function and an adapted optimization algorithm, satisfactory results have been obtained.

However, it is important to note that an adaptation has been made to work out the cut ratio objective function used. This adaptation, on one hand, takes into account the geographical distribution of the cells through the calculation of the standard deviation of the neighborhood distances, and on the other hand, avoiding the dominance of the zones having a very few number of cells by integrating the number of edges into the weight calculation formula and the cut of a zone. Other adaptations and improvements can be made to this objective function in order to obtain better results. In addition, it would be interesting to work on so called _standardized cost objective function_ [SHI 00], by making the necessary adaptations, and there by compare the results obtained.

# 8.5. Bibliography

[BAL 67] BALL H.G., HALL D., "Clustering technique for summarizing multivariate data", _Behavioral Science_ , vol. 12, p. 153-155, 1967.

[BEN 03] BEN-AKIVA M., BIERLAIRE M., "Discrete choice methods and their application to short term travel decisions", _Handbook of Transportation Science_ , Kluwer, 2003.

[DRÉ 03] DRÉO J., PÉTROWSKI A., SIARRY P., TAILLARD E., _Métaheuristiques pour l'optimisation difficile_ , Eyrolles, 2003.

[ERL 17] ERLANG K.A., "Solution of some problems in the theory of probabilities of some significance in automatic telephone exchanges", _Elektroteknikeren_ , vol. 13, 1917, translated to Post Office Electrical Engineer's Journal, vol.10, p. 189-197, 1918.

[FOR 65] FORGY W.E., "Cluster analysis of multivariate data: efficiency vs. interpretability of classification", _Biometrics_ , vol. 21, p. 768, 1965.

[FOR 92] FORTUNE S., "Voronoi diagrams and delaunay triangulations", _Computing in Euclidean Geometry, World Scientific_ , p. 193-233, 1992.

[FRA 57] FRASER A., "Simulation of genetic systems by automatic digital computers", _Australian Journal of Biological Sciences_ , vol. 10, p. 484-491, 1957.

[HOL 62] HOLLAND J., "Outline for a logical theory of adaptive systems", _Journal of the ACM_ , vol. 9, no. 3, p. 297-314, 1962.

[KEN 03] KENNETH E., Discrete Choice Methods With Simulation, Cambridge University Press, 2003.

[LAG 99] LAGRANGE X., GODLEWSKI P., TABBANE S., _Réseaux GSM-DCS_ , 4th edition, Hermès, 1999.

[MAC 67] MACQUEEN J., "Some methods for classification and analysis of multivariate observation", in LE CAM L., NEYMAN J. (eds), _Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability_ , Berkeley University Publisher, CA, p. 281-297, 1967.

[OUG 08] OUGHDI M., Régulation de la demande dans les réseaux mobiles par optimisation de la tarification, PhD Thesis, Université de Technologie de Belfort-Montbéliard, 2008.

[SHI 00] SHI J., MALIK J., "Normalized cuts and image segmentation", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 22, no. 8, p. 888-905, 2000.

[WEI 89] WEI Y., CHENG C., "Towards efficient hierarchical designs by ratio cut partitioning", _Proceedings of the IEEE International Conference on Computer-Aided Design_ , IEEE Publisher, p. 298-301, 1989.

[WEL 00] WELDON K., "A simplified introduction to correlation and regression", _Journal of Statistics Education_ , vol. 8, no. 3, p. 1-7, 2000.

* * *

1 Chapter written by Mustapha Oughdi, Sid LAMROUS and Alexandre CAMINADA.

1The utility function used for the behavioral model of clients is described in detail in the doctoral thesis of OUGHDI [OUG 08].

2 The concept of profile correlation is discussed later in this chapter.

# Chapter 9

# Air Traffic Control Graph Partitioning Application 1

# 9.1. Introduction

At present in Europe, the elementary control sectors, which are the smallest subdivisions of the airspace for a controller, are grouped into qualification areas: Figure 9.1 shows the example of the East qualification area of theAix en Provence center in France. Each air traffic controller is trained to work on all the elementary sectors of a qualification area, as well as grouped ones, but cannot change the qualification area without undergoing a long and specific training. Currently, there are 7 qualification areas in France, grouped together in the 5 control centers. Table 9.1 shows the list of control centers and the their qualification areas of several European countries which constitute to form, as we will discuss later, what we call as European _core area._

The Single European Sky project (SES), developed by the European Union, plans to create the Functional Airspace Blocks (FAB), which will subdivide the European sky according to the traffic flows and no longer according to the national borders, as followed these days. This project plans to set up FABs in the core area, which is defined as the European airspace region with the dense traffic. This notion is often used by Eurocontrol1 in its examples, but without ever defining the countries or control centers it contains. In order to face this problem, the study described in [BIC 05] estimates that the dense air traffic in Europe is intense in the following countries: Spain, Italy, Switzerland, Austria, Belgium, Germany, Luxembourg, the Netherlands, Denmark,

**Figure 9.1.** _Projection of the second Aix en Provence qualification area at flight level FL 300 (altitude of 9,144 m)_

France, and the United Kingdom. These 11 countries consititute the European core area.

This chapter summarizes the part of the work carried out by Charles-Edmond Bichot in his thesis [BIC 07a, BIC 07b] regarding a new graph partitioning optimization method and whose major application concerns with the optimization of control qualification areas in the European airspace. The main objective of the thesis is to construct a graph that models the existing traffic flows between the control center in Europe, and to regroup the sectors into the qualification areas, thereby minimizing the flows between the two different controlled qualification areas and maximizing the flows within a single qualification area. This simple criterion minimizes the workload coordinated between the distinct qualification areas.

The obtained results obviously cannot be utilized directly, but rather allow the observation of several phenomena showing the structure of the European air traffic, in particular, its flow organization, and form several potentially useful conclusions to bring about a change in the current organization of qualification areas.

This chapter is divided into 4 complementary sections. Section 9.2 provides the details on the context of the study. Section 9.3 proposes a model of the problem and the optimized criteria. Section 9.4 introduces an optimization method of the problem, which has proved to be particularly efficient, not only on the problem of the optimizing FABs, but also on the whole category of graph partitioning optimization problems. This

**Table 9.1.** _Distribution of sectors, areas, and control centers on June 2005 in the 11 countries of the European core area_

method is compared to other classical optimization tools. Finally, section 9.5 presents the results obtained from the problem of creating functional blocks in central Europe. These results, which cannot be used just as they are, allow some conclusions on the organization of the traffic flows in Europe, and can be used as a basis for considerations regarding the evolution of the current organization of the qualification areas toward the Single Sky objective.

# 9.2. The problem of dividing up the airspace

The airspace is divided into qualification areas that are, themselves, a part of enroute control centers. This means that there are already several divisions of the airspace in Europe, depending on whether we are talking about the division into sectors, areas, or centers. Table 9.1 shows a few characteristics of these different divisions.

For practical reasons, en-route control differentiates the upper airspace from the lower airspace. In the European parliament regulations, the division level between the upper and lower airspaces is fixed at a flight level 285 (8,687 m) [REG 04a]. However, from an operational point of view, the upper airspace is positioned above the flight level FL195 (5,944 m) and the lower airspace is below the flight level.

This operational differentiation is based on the cruising flight level of commercial airplanes. This is positioned above FL195, while when aircraft is not cruising, they are in the ascent or descent phases of flight. The resulting air traffic control and the related problems are therefore different. Despite of the interest in the operational differentiation, the differentiation introduced by the European regulations will be used in the future.

From European Sky's point of view, the current study concerns the reorganization of the control sectors into the qualification areas in future, in the sense with the creation of functional airspace blocks according to the European Commission. The control sectors concerned in our study are those of the qualification areas, therefore signifying the upper and lower airspace sectors of the en-route control centers. From the qualification areas that are proposed in another study, it is possible to create the corresponding en-route control centers, which constitute a group of one or more qualification areas.

## 9.2.1. _Creation of functional airspace blocks in Europe_

The creation of functional airspace blocks in Europe is a part of the Single European Sky framework.

The European Commission adopted on October 10, 2001 is a set of measures on the airspace traffic management with the aim of being able to develop the regulatory framework of the Single European Sky by the end of 2004. The purpose of the Single European Sky is to put an end to the fragmentation of the airspace of the European Union and to create a borderless airspace that is safe and efficient. The result of this work was a framework regulation [REG 04b] associated with three regulations [REG 04c, REG 04a, REG 04d]. These regulations provide an idea to create the functional airspace blocks in the upper airspace, above the European Union. In the framework regulations, a functional airspace block is defined as "which is based on the operational needs that express the necessity to ensure a more integrated management of the airspace, independent of the existing borders".

The regulations insist on the fact that the reconfiguration of the airspace should be based on the operational requirements, independently of existing borders. In addition, the general principles linked to the creation of the functional blocks should be elaborated by the air traffic service providers of the different member states, in consultation with Eurocontrol and based on its technical advice. The creation and the modification of the functional blocks are responsibile for the concerned member states. The regulation requires the functional blocks to be set up by the year 2009.

Concerning the reconfiguration of the upper airspace (above FL285) into functional blocks, two conditions that are necessary for their creation affect, in particular, our application in the airspace division are as follows:

– the functional blocks "permit the optimum use of the airspace, taking into account the air traffic flows". The air traffic flows management aims to contribute a completely safe, organized, and fast air traffic flow;

– the functional blocks are justified by the global added value that they provide.

In order to set up the general principles linked to the creation of functional blocks, seminars have been organized by Eurocontrol inviting the different suppliers of air navigation services across the European Union. The three consultations were presented in 2004 and at the beginning of 2005. The purpose of each of these meetings was to validate the work accomplished by various teams on the problems linked to the creation and operation of the functional blocks. An advisory report for the European Commission summarizes the work accomplished, and set out by these seminars [EUR 05]. During these seminars, it was decided that the functional airspace blocks should:

– be designed based on the operational requirements;

– create a more coherent air traffic airspace;

– not taking the national borders into account.

The functional airspace blocks must be designed to optimize the air traffic flows. However, it was also accepted that the creation of functional airspace blocks was a long and complex process.

In [REG 04a], it is stated that the functional airspace blocks must respect 7 criteria, which are as follows:

1) the FABs are supported by a safety file;

2) they allow optimum use of the airspace, taking into account the air traffic flows;

3) they are justified by the global added value that they provide, including the optimum use of technical and human resources based on the cost-efficiency studies;

4) they ensure a fluid and flexible transfer of air traffic control responsibility between the air traffic service units;

5) they ensure the compatibility between the upper and the lower airspace configurations;

6) they comply with the conditions resulting from the regional agreements drawn up with the ICAO;

7) they comply with the regional agreements that exist on the date when these regulations come into effect [. . . ].

These criteria are sometimes difficult to interpret. They are too general to be perfectly modeled (such as the criteria 2, 3, 4, and 5), or they simply cannot be modeled in the framework of our problem (this is the case of the criteria 1, 6, and 7). Similar structures to functional blocks have already existed in Europe. For example, this is the case of the airspace area controlled by the Eurocontrol center in Maastricht. This center has been in charge of controlling the upper airspace over Benelux and north-west Germany since 1972. It has also been the case since 1997 in the east of Europe with the creation of the Central European Air Traffic Services which groups together the airspaces of Austria, Hungary, Czech Republic, Slovakia, Slovenia, Croatia, Serbia, and north-eastern Italy. The official and multinational structure of the functional blocks will facilitate and generalize the procedure to create the common control areas like Maastricht or the one in eastern Europe, and ensure their perpetuation. This study has taken into account all the sectors of the en-route control centers, whether they are upper or lower. Although the regulation currently provides the creation of functional blocks in the upper airspace, the following three reasons explain this standpoint:

– few current qualification areas are divided according to FL285, and changing the qualification area to FL285 may negatively affect the traffic fluidity;

– the aim of this study is not to propose an immediate operational solution, but to give the general indications on the influence of certain criteria on the structure of future functional airspace blocks;

– in article 10, the regulation [REG 04a] also provides the possibility of extending the functional blocks to lower the airspaces. It would seem that this possibility is envisaged by most of the air traffic service providers from various European states. In this case, according to the regulation, the functional blocks would incorporate all the sectors of the en-route control centers without any distinction.

## 9.2.2. _Creation of a functional block in central Europe_

A feasibility study has been made concerning the creation of a functional block at the heart of Europe: the central European FAB. Six countries are associated for this project which are as follows: Germany, Belgium, France, Luxembourg, the Netherlands, and Switzerland. This study ensues from the likeness between the functional block projects of France and Switzerland, as well as Germany and Benelux countries. These two functional block projects were prepared between 2005 and 2006. The aim of the central European FAB is to set up a functional block incorporating the upper and lower airspace of these 6 states, as well as to find the common solutions oriented toward the performance, and as stay independent of national borders [FAB 07]. The area covered by this functional block is one of the densest in the world in terms of traffic and is characterized by the civil and military airways that are closely overlapping. The feasibility study should finish at the end of 2008 in order to respect the deadline for the creation of functional blocks anticipated for 2009. The expected advantages of such functional blocks are as follows::

– complete absorption of the anticipated traffic growth;

– optimum use of the airspace according to the traffic flows;

– optimum use of human and technical resources;

– perfect compatibility between the upper and lower airspaces;

– high-performance economic level of service;

– reinforced civil-military cooperation.

The procedure that was started by the central European FAB could include rethinking of the civil and military airspace independently of national borders. As in the case of the previous subsection, the sectors concerned by this boundary change are those of the en-route control centers in the various countries concerned.

# 9.3. Modeling the problem

Before modeling the problem, it might be helpful to recall a few notions in order to understand the workload of an air traffic controller better.

## 9.3.1. _Control workload in a sector_

A sector is controlled by a pair of controllers, who ensure the safety of flights that cross it by separating the aircraft from each other according to the standards in force. The larger the number of aircraft in a sector, the more the control workload increases in a non-linear way. There is a limit beyond which the controller can no longer accept new aircraft, so he makes them fly through less loaded neighboring sectors. The sector is said to be saturated. This critical situation must be avoided as it provokes a cumulative phenomenon of overload on the sectors upstream, and may even go as far as the departure airports.

In the controller's workload, three main components are usually distinguishable, which are as follows:

– _Monitoring workload_ : in a sector, apart from any actions taken on the trajectory, the controller must check that the flight plans are proceeding correctly on the radar image, and determine the potential risk of the future conflicts caused by the surrounding aircraft.

Monitoring is the controllers' basic task, but it is a major source of stress for them. This workload is directly linked to the number of aircrafts in the sector:

– _Resolution workload_ : when a risk of conflict is detected, the controller can act on the aircraft trajectory in order to maintain the minimal separation standards;

– _Coordination workload_ : when an aircraft changes the sectors, the controller who was in charge of it makes a control transfer and therefore the aircraft must change its radio frequency. Earlier, the transfer must have been accepted by the controller, who is receiving the aircraft. An agreement is made between the two controllers, the one receiving and the one transferring, in order to ensure that the aircraft can be accepted and to define the transfer procedures such as the flight level, heading, and so on. The coordination workload is proportional to the flows cut by the sector borders.

## 9.3.2. _Objective: minimizing the coordination workload_

Elementary sectors can only be grouped together inside a qualification area. As a result, it is a lot to our advantage to limit the flows between the different qualification areas that systematically require coordination, whereas for several hours each day some of these coordinations disappear inside a qualification area. We can expect that the coordination task is easier between the two sectors in the same qualification area, as the controller who performs it has a good understanding of both the sectors. This, however, is not the case when he carries out a coordination with another qualification area [HAL 05].

There are two objectives that can decrease the controller's coordination workload, namely:

1) minimizing the coordination workload between the two different qualification areas. To do so, the most efficient solution is to minimize the number of aircrafts passing through the different qualification areas;

2) maximizing the number of aircraft between the sectors of a same area, as the coordination workload of these flows, is lower.

The monitoring and the resolution workloads are not, in principle, affected by the reorganization of the qualification areas.

## 9.3.3. _Two constraints, the size of the qualification areas and size of the control centers_

It is very difficult to know the optimum number of the sectors per qualification area. Some feedback provided by air traffic controllers would set this number between 22 and 28, this order of magnitude is nevertheless subjective. The average number of sectors per control center is currently 9 for the 38 member states of Eurocontrol. This number is 37 in the United States. Studies have been carried out which point out these differences [EUR 06a, EUR 06b]. With the aim of optimizing costs, they recommend the creation of control centers with an optimum number of 25 sectors per center. According to these studies, the number should not exceed 50-60 sectors for a single center, a threshold beyond which the scale savings are caught up by the costs related to the size of the center. Figure 9.2 shows the variation in cost per sector, according to the size of the center. By adopting a configuration of 25 sectors per center, the forecast savings compared to the current system are 20% for the operating costs and 25% for the investment costs.

**Figure 9.2.** _Variation of the cost per sector according to the size of the center [EUR 06a]_

Since the creation of functional blocks planned by the Commission is based on the cost/efficiency study [REG 04a], our study will assume that a qualification area must preferably contain 25 sectors, which is within the estimated range from the feedback, but in agreement of the Eurocontrol recommendations, the maximum number could achieve double of this.

The other important criterion is the control workload of each area. A qualification area may include many sectors if the average number of aircrafts per sector is not too high. If this average number of aircrafts is high, then the qualification area must be of smaller size, because when the number of aircraft increases, the number of routes and/or procedures to know increases too. In these cases, the controllers have to control each sector more frequently in order to remain trained on them, in the sense that for the same workload, they will know less about them. Therefore, it is very important to keep a certain balance between the various areas regarding the number of aircraft that cross each of them. Furthermore, an imbalance in the control workload between the areas would be prejudicial for the controllers in the busiest areas.

## 9.3.4. _Analysis and processing of European air traffic data_

The data used are flight plans filed with the Eurocontrol's CFMU2. Three days of traffic on June 2005 (Friday 17 and 24 June, and Monday 20 June) and about a 100 days spread between July 2001 and September 2002 were provided by the Eurocontrol's 2 The experimental center. The data of the year 2005 (particularly that of Friday 17 June) were used in priority for this first exploratory approach as the data was collected recently.

The flight plan data were provided per UTC day3 in files summarizing the flight numbers and passage times of aircrafts in the sectors crossed, as well as the names of these sectors. The flight plan data is provided with a description of the 3D coordinates for each sector. After processing, these coordinates, sometimes imprecise or incomplete, allow the creation of sector maps and qualification areas at the corresponding dates.

The reason for the analysis of European flight plans is to obtain a daily database of air traffic flows between the different European sectors. In this way, for each sector in Europe, we know the number of aircraft having filed a flight plan that crossed it, as well as the flows between the sector and each of its neighbors.

The flight plan data provided by Eurocontrol had to be processed before being used. The most frequently encountered problem concerns with the designation of the sectors, because numerous sectors shown on the flight plans do not appear on the nomenclature of the associated sectors. Another recurrent problem concerns with the validity of the written travel times. Certain flows do not depart from or arrive at a known sector. This is the case of the aircraft taking off from the small airfields, or even that of a landing aircraft. It is also the case of aircraft leaving the area covered by the sector description of the Eurocontrol database. Therefore, an aircraft going to Canada will leave Europe, but will still be counted in a flow leaving the last European sector it crossed.

## 9.3.5. _Graph of European air traffic and adaptation to partitioning_

The daily database of aircraft flows between the different European sectors obtained from the analysis of the European flight plans, allowed the creation of a European air traffic graph. To do this, as illustrated in Figure 9.3, each control sector must be considered as a vertex of this graph. Moreover, for each flow between the two sectors, an edge links the two corresponding vertices, weighted for the number of aircraft of this flow. Some flows do not correspond to the flows between the sectors. Nevertheless, these are still taken into account in the weight of a vertex. Therefore, the weight of a vertex corresponds to the sum of the number of aircraft of all the flows linked to the sector corresponding to this vertex.

This also means that the weight of a vertex is not equal to the sum of the weights of all the edges linked to this vertex. However, the weight of this vertex has a very concrete meaning as it corresponds to the number of coordination tasks carried out

**Figure 9.3.** _The modeling process to divide the airspace. In this example, it is illustrated on the French sky at FL300 for a partitioning into 6 areas_

in the corresponding sector during one day. In the same way, the weight of an edge corresponds to the number of coordination tasks carried out by the controllers in the departure sector or the ones in the arrival sector. Like this, the flight plan database allows the creation of a graph. Therefore, the problem of the European airspace division becomes a graph partitioning optimization problem.

Let a graph _G_ be composed of a set of vertices _V_ and edges _E_. A partition of _G_ is a set of vertex subsets in disjoint pairs, such that their union is equal to _V_. The graph partitioning optimization problem consists of finding a partition of _G_ in _k_ parts which minimizes a cost function and complies with the constraints of the problem.

At this stage, a few definitions may be useful to state the optimized criteria, namely:

– the weight of a part or a qualification area equals to the sum of the weights of the vertices or sectors that compose it;

– the average weight of the partition is the total weight of the graph vertices divided by the number of parts of the partition;

– the balance of the partition is equal to the ratio between the weight of the most weighted part and the average weight of the partition. The resulting partition balance allows a measurement of the imbalances between the qualification areas in terms of controlled flows;

– the cut between the two parts of the partition is the sum of the weight of the edges linking the vertices not belonging to the same part. Regarding the air traffic control, the cut between the two qualification areas is the sum of the flows crossing the two qualification areas;

– the cut of the whole partition is the sum of the cut of all the pairs of parts forming the partition. Regarding the air traffic control, the division cost of the airspace is the sum of all the flows going from one qualification area to another;

– the ratio cut of the partition is the sum of each part of the partition of its cut divided by its weight. Regarding the air traffic control, it is the sum of the ratios of the qualification areas coordinating toward their neighboring areas and their internal coordination workload.

It is this last criterion that is optimized as it simultaneously optimizes both the objectives fixed in a section 9.3.3. The problem constraints are linked to the size and the weight of the qualification areas. These are constraints on the partition balance. As we have seen, a qualification area must have an average of 25 sectors and a maximum of 50 sectors. This relatively significant value was deliberately chosen, as the weight of an area is far from being the only indicator of control difficulty, but beyond twice the number of coordinations in an area, this indicator really becomes very large indeed. More practically, the airspace partitions will always be proposed that have a partitioning balance which is either close to the actual division if this is less than 2.0, or more than 2.0 in the opposite case.

# 9.4. Airspace partitioning: towards a new optimization metaheuristic

Because of its large size and difficulty, the graph partitioning optimization problem presented above cannot be solved in an optimum manner by using the classical tools, which are found in the specialized literature in both the areas of graph partitioning and integer programming. On the basis of the stochastic optimization methods, several approaches were tested in order to solve the problem. The approaches based on genetic algorithms or simulated annealing were considered. Also the average results were obtained, and the specificity of the problem led us to consider a new approach, which has analogies with nuclear physics and is named as the fusion-fission.

At the beginning, this method was specifically created to solve the problem of the European airspace division, and then it evolved to become a fully-fledged graph partitioning meta-method. This method was named as the fusion-fission, because the model used as a basis for its development was from the area of nuclear physics, and more specifically the fusion and fission of atoms.

The purpose of this section is to describe simply and concisely the fusion-fission algorithm, which was created to solve the problem of the airspace division. A more recent version of the fusion-fission algorithm which is adapted to the constrained graph partitioning optimization problem, in the sense minimizing the cut cost with a strong constraint on the balance of the partition, is discussed in Chapter 7.

The fusion-fission is a method inspired by the real world. This source of inspiration is often used to solve the combinatorial optimization problems, notably owing to metaheuristics. More specifically, the fusion-fission method is based on an analogy with nuclear physics, and its fusion and fission mechanisms.

The fusion process is illustrated in Figure 9.4. In this process, the two atoms fuse together to create a new atom having as many nucleons as the sum of the two parent atoms, minus a certain number of nucleons that will be ejected during this process (this number depends on the nature of the reaction).

**Figure 9.4.** _Process of fusion_

**Figure 9.5.** _Process of fission_

Figure 9.5 describes the fission process. Here, an atom is split into two new atoms not necessarily of the same size. During this process, certain nucleons can be ejected. Here again, the number of the nucleons depends on the type of the reaction.

The analogy proposed with the nuclear fusion and fission processes have each nucleon corresponding to exactly one vertex of the air traffic graph (i.e. one control sector). It follows that the weight of an edge linking the two vertices (i.e. the flow of aircraft between the two sectors) will correspond to the binding energy between the two nucleons, which is physically conveyed by their connection. Then, we consider that each atom is an analog to a part of the sought partition. In this way, we have an adaptation of the fusion-fission mechanism to the graph partitioning optimization problem, which itself models the airspace division problem.

In a general manner, the fusion-fission algorithm iterates a sequence of three consecutive actions:

1) select one of the parts of the partition;

2) according to the characteristics of this part and the partition, choose the process (fusion or fission) which will be suitable;

3) apply this process (in the case of the fusion, another part will be selected too).

Important points resulting from this sequence of actions are as follows:

– the number of parts of the partition varies either by plus one or by minus one at each iteration;

– the number of parts must be limited in order to stay within the limits of the problem;

– a nucleon ejection mechanism must be used during the fusion and fission processes.

The nucleon ejection mechanism used is coupled with a learning method which allows the adaptation of the quantity of the nucleons that are ejected during the progression of the algorithm, and follows the quality of the partitions found.

**Figure 9.6.** _Fission processes applied to a fictive division_

The process of fission splits a part into 2 parts; an algorithm based on the mechanism of percolation is used for this task.

The percolation concerns with the movement of a fluid that flows gradually from one place to another: the fluid leaves an initial spot and spreads. In the example of graph partitioning, the fluid has to spread throughout the mesh in the graph, flowing from _k_ initial vertices. These _k_ vertices form the premises of the _k_ parts of the final partition. In order to distinguish the various parts from each other, a different color of fluid flows from each initial vertex. By gradually spreading throughout the graph, the fluid moves across the vertices and gives them its color. The intensity of the fluid decreases proportionally with the distance it has flowed from the initial vertex, and is inversely proportional to the sum of the weights of the edges of the path taken. The process becomes more complex when the fluids meet. The fluids are not able to mix, in the sense that one fluid cannot penetrate the space of the other fluid, it can only push

**Figure 9.7.** _The fusion process applied to a fictive division_

it back so as to reach the vertices. On a vertex that several fluids are competing for, the fluid of greatest intensity succeeds.

Figures 9.6 and 9.7 show, in the case of a fictive division, an iteration of the fusionfission algorithm for which the selected area will be subjected to the fission and fusion processes, respectively.

# 9.5. Division of the central European airspace

The central Europe functional block project will group together 6 European countries (section 9.2.2): Germany, Belgium, France, Luxembourg, the Netherlands,

**Table 9.2.** _Results of dividing the central Europe FAB airspace into 12, 16, and 26 parts_

and Switzerland. These 6 countries currently possess 16 en-route control centers and 26 qualification areas, for a total of 300 sectors. By applying the Eurocontrol recommendations, the central Europe FAB should possess 12 control centers. For example, we propose in this section, three divisions of the central European functional airspace block. The first is divided into 12 parts, the second into 16 parts, and the third into 26 parts, each of them representing a possible division into qualification areas of the central European FAB. For each of these three divisions, we compared the results obtained by using the different algorithms and partitioning tools. We then compared the results obtained by using the fusion-fission algorithm with those found using two classical partitioning techniques: the Scotch and Graclus software (see the Appendix). Although the Scotch software seeks to minimize the cost of cutting a partition, the Graclus software seeks to minimize the ratio cut, i.e. the fixed objective function.

Table 9.2 gives a summary of the characteristic values of the existing or created divisions of the central Europe FAB airspace. The table is divided into 3 parts:

– the part 1 of the table shows several divisions into 12 parts. The result obtained by using fusion-fission method has the best cutting and ratio cut, and is similar to the Scotch tool for the partitioning balance. The Scotch tool has a slightly lower maximum number of sectors (32) but, in all three cases, this number is far lower than the limits proposed by Eurocontrol (50). Lastly, with a minimum number of 17 sectors, here again the fusion-fission show the better results. The most interesting partition, therefore, seems to be provided by the fusion-fission;

– the part 2 of the table compares the real division into 16 control centers on 17 June 2005, with the partitions into 16 parts found by using the other three methods. The cut and ratio cut of the real division are very close to those in the partition found by using the Graclus tool. So at first sight, we might think that the real division needs a little improvement. However, the Graclus partition balance is far lower than that of the real division (1.42 versus 1.91); the decrease in the balance of a partition goes hand in hand with the increase in its ratio cut. Concerning the rest of the results, it is the partition found by the fusion-fission which is the best, although with a partitioning balance that is slightly higher than the partition found by Scotch;

– the last part of the table shows the characteristics of the real division into 26 qualifications areas and the partitions found using the three partitioning methods into 26 parts. The partitioning balance of the actual division is so high that we can expect the qualification areas of these 6 countries to be of unequal control difficulty. Again, in this case, it is the fusion-fission algorithm that found the lowest partitions values of the ratio cut, cut, and balance.The maximum and minimum number of sectors are very low for all the partitions found, compared to the recommendations obtained from the controllers. This is because of the very high number of current qualification areas in these 6 countries.

Figure 9.8 illustrates the division found by using the fusion-fission method to cut the central European FAB airspace into 12 parts. By comparing these results with the current division into 12 control centers (Figure 9.8(a)), several similarities come into view. At lower flight levels, the parts of the partition located to the west, south-west, and south of France are found in both figures, as well as the eastern part of Germany. At medium flight levels, certain parts of the partition found, by using the fusion-fission, which seem to regroup together two existing centers: the Netherlands, north-west Germany, east Switzerland, and east of France. At upper flight levels (above FL285), the correspondence between the centers and the parts are clearer above Germany and Benelux, with for the center of Benelux (which is in fact that of Eurocontrol) a division into 2 parts in the fusion-fission result. It can be observed at the upper levels that in Belgium the fusion-fission division shows a highly marked part at its borers, even though this country is located right in the middle of the division. The division studied is based on the sectors that were, themselves, created based on the national borders and therefore any new division based on these sectors will necessarily be biased by the shape of these sectors. In addition, this new division is based on the existing air routes. In order to be as efficient as possible, a new division would require a modification of these air routes, as well as the geography of these sectors.

During this study, a division of the French airspace into 26 parts was also made. The results obtained from the fusion-fission method on the French airspace are very similar, concerning the French regions, to the division of the central European FAB

**Figure 9.8.** _Division into 12 parts of the central European FAB, obtained by the fusion-fission algorithm. These images are in color on the Website:perso.ec-lyon.fr/charles-edmond.bichot/_

into 12 parts found by using the same method (Figures 9.8(a) and 9.8(b)). At lower flight levels, the only part which differs is the north, which is divided into two and overflows to the south (as shown in Figure 9.8(a)). At upper flight levels, the same part of the north of France is divided into two (as shown in Figure 9.8(b)), one part goes toward the east-Switzerland center and the other toward the center of France.

Figure 9.9 shows the actual division of the central European FAB into 16 control centers, with the partition into 16 parts found by using the fusion-fission method on the same airspace. At lower levels, again there are many similarities in between the division found by the fusion-fission, the division found for France, and the one found for the central European FAB (figure 9.9(b)). In both these results, Paris is in the center of a star formed, by the French control centers in one case, and by the French parts of the partition in the other. The resemblance is still very strong in the case of the upper flight levels (Figure 9.8(d)). This observation is a positive point for the fusion-fission method, which allows the homogeneous solutions to be found, whereas the difficulty increases.

On the other hand, if we compare the actual division of the central European FAB control centers and the partition into 16 parts found by the fusion-fission method, then we can observe that these two division are quite different. The greatest similarities observed concern the western France and southern France parts, for all the flight levels. It is interesting to note that at lower flight levels, the German parts found by the fusion-fission form a star around Frankfurt, like the case of Paris in France. The phenomenon

**Figure 9.9.** _The images on the left, (a) and (c), show the 16 control centers of the central European FAB. Those on the right, (b) and (d), propose a division into 16 parts of this airspace calculated by the fusion-fission algorithm. These images are in color on the Website of this book:perso.ec-lyon.fr/charles-edmond.bichot/_

disappears at the upper flight levels. It can also be observed that at the upper flight levels, the division follows more than the state borders at the lower flight levels.

Figure 9.10 shows the actual division of the central European FAB into qualification areas. With 26 qualification areas, the average number of 12 sectors per area is very low. It is, therefore, not surprising that the division found by the fusion-fission

**Figure 9.10.** _The images on the left, (a) and (c), show the 26 qualification areas of the central European FAB. Those on the right, (b) and (d), propose a division into 26 parts of this airspace calculated by the fusion-fission algorithm. These images are in color on the Website of this book:perso.ec-lyon.fr/charles-edmond.bichot/_

(Figures 9.10(b) and 9.10(d)) contains a large number of different parts. However, the most important observation in this partition is that the number of parts present at lower flight levels is much higher than that of the upper flight levels. This corresponds to the type of the traffic at different flight altitudes. This is because the lower flight levels are in contact with the approach regions of the large airports, and their purpose is to make the aircraft ascend or descend. At these flight levels, the parts therefore tend to be very high, but have a smaller base area. However, at upper flight levels, the aircraft are all at cruising altitude, which results in an increase in the areas of these parts, but this also flattens them out. This explains why we can more easily find the structured parts around the traffic routes in the vicinity of the large airports, or between them, at lower flight levels, and wider parts at the upper flight levels.

Of course, the results obtained from these three examples cannot be used directly and can only give indications regarding the influence of the optimized criteria on the creation of FABs in the central Europe.

We have seen that this criterion leads to the creation of fewer, wider, qualification areas in the upper airspace, and numerous areas covering more flight levels in the lower airspace. The organization of the qualification areas in the lower airspace should preferably be in star formation around the main European hubs. Finally, we observe that the national borders remain very apparent, and that the results obtained are not as far from the current reality as we could have expected. This leads us to think that the current route network and traffic flows are, today, highly dependent on the organization of the European sky, and the current qualification areas. A reorganization of the qualification areas will probably lead to an evolution in the traffic flows, which leads us to think that the problems of sectorization, routes network, flow organization, and qualification areas should be treated simultaneously, if possible.

# 9.6. Conclusion

From this study, we have been able to show how complex it is to model the organization of the European airspace into FABs. In addition, the European Commission's recommendations are difficult to interpret, and few criteria are modeled simply. Therefore, in this study, we attempted to minimize the coordination task between the sectors of different FABs. This simple criterion generates a graph partitioning optimization problem that is difficult to solve, and for which classical graph partitioning methods do not always provide satisfactory solutions. A new heuristic based on a combinatorial optimization technique, named as the fusion-fission, was adapted to this problem. Also, there is no doubt the data samples used have a heavy significant influence on the results obtained. Although we limited this study to a few days of traffic, it would be useful to test the sensitivity of the results obtained from the samples used.

The purpose of this study is not to propose a directly operational division of the European airspace into the qualification areas, but to propose tools that allow guiding of such a division and may be to assess certain choices. For example, it can be observed without any doubt that the organization of the FABs in a star shape around the main European hubs in the lower airspace would be a solution limiting the coordinations between the qualification areas. In the anticipation of obtaining the operational results, it would be important to carry out a field study concerning the difficulties relating to each region of the airspace.

The results obtained show that the proposed divisions often restore the national borders. We could therefore suppose that a simultaneous optimization of the route network and the FABs could permit the significant modification of the results obtained, with the risk of making the problem even more complex to solve.

# 9.7. Acknowledgments

This work was financed by the Direction Générale de l'Aviation Civile (DGAC, French Department for Transport) and in particular the Direction Technique et Innovation de la Direction des Services de la Navigation Aérienne (DSNA/DTI). We would like to thank in particular Jean-Marc Alliot, Jean-Baptiste Gotteland, David Gianazza, Kevin Guittet, Pascal Brisset and Nicolas Barnier for their comments and support.

# 9.8. Bibliography

[BIC 05] BICHOT C.-E., ALLIOT J.-M., A theoretical approach to defining the European core area, Report, École Nationale de l'Aviation Civile / Center d'Étude de la NavigationAérienne, France, 2005.

[BIC 07a] BICHOT C.-E., Élaboration d'une nouvelle métaheuristique pour le partitionnement de graphe: la méthode de fusion-fission, Application au découpage de l'espace aérien, PhD Thesis, Institut National Polytechnique de Toulouse (INPT), November 2007.

[BIC 07b] BICHOT C.-E., "A new method, the fusion fission, for the relaxed k-way graph partitioning problem, and comparisons with some multilevel algorithms", _Journal of Mathematical Modeling and Algorithms (JMMA)_ , vol. 6, no. 3, p. 319-344, 2007.

[EUR 05] EUROCONTROL, Final report on European commission's man date to support the establishment of functional airspace blocks, May 2005.

[EUR 06a] EUROCONTROL, The impact of fragmentation in European ATM/CNS, Report, Eurocontrol, April 2006.

[EUR 06b] EUROCONTROL, Performance Review Report, Report, Eurocontrol, Performance Review Commission, April 2006.

[FAB 07] FAB Europe Central – Redefining air traffic control in the heart of Europe, jointproduction of Belgocontrol, DSNA, DFS, LAA, LVNL, MUAC and skyguide, 2007.

[HAL 05] HALLGREN A., "Restructuring European airspace: functional airspace blocks", _Skyway_ , Eurocontrol Publication and Information Center, Autumn, p. 20-22, 2005.

[REG 04a] REGULATION (EC) No 551/2004 of the European Parliament and of the Council of 10 March 2004 on the organization and use of the airspace in the single European sky (the airspace Regulation), _Official Journal of the European Union_ , 2004.

[REG 04b] REGULATION (EC) No 549/2004 of the European Parliament and of the Council of 10 March 2004 laying down the framework for the creation of the single European sky (the framework Regulation), _Official Journal of the European Union_ , 2004.

[REG 04c] REGULATION (EC) No 550/2004 of the European Parliament and of the Council of 10 March 2004 on the provision of air navigation services in the single European sky (the service provision Regulation), _Official Journal of the European Union_ , 2004.

[REG 04d] REGULATION (EC) No 552/2004 of the European Parliament and of the Council of 10 March 2004 on the interoperability of the European air traffic management network (the interoperability Regulation), _Official Journal of the European Union_ , 2004.

* * *

1 Chapter written by Charles-Edmond BICHOT and Nicolas DURAND.

1Eurocontrol is the European organization for the safety of air navigation.

2 The CFMU, Control Flow Management Unit, is the section of Eurocontrol charged with coordinating the circulation of air traffic in Europe, in order to avoid the congestion and ensure that the available control capacity is used correctly.

3UTC is the acronymn of Coordinated Universal Time. This is an international time scale.
PART 3

Other Approaches to Graph Partitioning

# Chapter 10

# Application of Graph Partitioning to Image Segmentation 1

# 10.1. Introduction

Over the last few years, a significant amount of work has been published on the application of graph theory to image processing, in particular image segmentation. In this chapter, we will discuss the use of graph partitioning for this purpose. Our aim is to introduce the modeling of an image in the form of a graph, and then to explain several methods of image segmentation, with examples that illustrate the performance of these approaches.

In section 10.2, we explain how to obtain a graph from an image. In the following section (section 10.3), we will introduce different applications of graph theory in image segmentation. The next section (section 10.4) is devoted to the application of graph partitioning to image segmentation. Then an original approach combining several segmentation techniques based on graph theory – an approach called _power watersheds_ – is introduced in section 10.5.

# 10.2. The image viewed in graph form

Generally, an image can be represented by an undirected graph. The approach used in this case, as illustrated in Figure 10.1, uses a _pixel adjacency graph_ [ROS 70, ROS 83]. It works on the following principle: the pixels of the image are the vertices of the graph, and the arcs connect each pixel to its "neighbors". In essence, to construct

**Figure 10.1.** _Pixel adjacency graph_

**Figure 10.2.** _Examples of several of the most common neighborhood rules used in 2D image processing_

the graph, we require an image and its adjacency matrix. Figure 10.2 shows several types of neighborhood policies, which are used during the construction of an adjacency graph.

In 3D applications, other types of neighborhood are used, such as neighborhood 12 or neighborhood 18.

**Figure 10.3**. _Illustration of low level segmentation_

**Figure 10.4.** _Example of the result of image segmentation ofFigure 10.3(a), using WT_

Another interesting approach to represent an image in graph form is the region adjacency graph, which is also called region graph [ROS 82]. In this case, the set of graph vertices corresponds to several regions of the image, for example a pixel class, and the arcs connect the neighboring regions. A region graph can be obtained by using low level segmentation. This low level segmentation (over-segmentation) can be obtained by using the application of non-supervised image classification [MCQ 67, NAK 09] or a "watershed" transformation (WT) (Figure 10.3) [BEU 93, VIN 91, NAJ 96, COU 09b]. The aim of the low level segmentation is to create homogeneous regions which replace the pixels, since these pixels drastically increase the complexity of the method. An example of image segmentation of Figure 10.3(a), using numerous WT methods, is shown in Figure 10.4.

The quality of this first stage is very important, since it allows us to ensure that low loss of information is low. An ideal low level segmentation allows us to highlight only relevant information present in the image. At the end of this non-supervised segmentation, each region is considered as a vertex, and the neighboring regions are connected by using a given adjacency system. The graph constructed in this manner contains fewer vertices and arcs, as compared with the one previously constructed.

**Figure 10.5**. _Example of a region graph_

This property allows a reduction in the computation time of the partitioning algorithm. An example that illustrate a low level over-segmentation, is shown in Figure 10.4. In the following section, we will discuss the usage of a region graph (Figure 10.5) and introduce some of the algorithms which use this approach.

There are other types of graphs to represent an image, for example the Delaunay dual graph [KOL 04]. A study of different types of graphs, in terms of the possibility merging the regions between themselves, is found in [COU 08c, COU 08b].

# 10.3. Principle of image segmentation using graphs

In this section, we will introduce several supervised segmentation methods by using the graph theory. In this supervised approach, the user specifies the zones, which are part of the concerned region of the image. It comes into play to set out (mark) the regions which must be segmented in the image. This zone location is carried out by using the markers. These markers correspond to one or several connected component(s), so that we can locate the regions. The automatic production of these markers is a central problem in all watershed based (WT) image segmentation algorithms [BEU 93, MEY 10].

The placement of these markers depends on the type of problem to be solved. Their number and their position can be fixed or variable. During initialization, they are placed randomly, in the case of an non-supervised method, or manually, in the case of a supervised method, or according to a marking.

The use of these markers is the simplest method to add a bias to the segmentation.

Marker-based image segmentation has become a standard approach, and forms an important stage in several segmentation approaches, such as region growing [ADA 94, BEA 06], watershed transform [BEU 93, MEY 02], optimal spanning forest [COU 09b], optimal path [FAL 04, XUE 07], random walkers and maximum flow [APP 06, BOY 03, GRA 07] methods.

The basic tool of image segmentation via a graph is the pixel graph of an image. In fact, the image segmentation generally results from a minimum spanning tree or the minimum partition separating the markers. In other words, the markers allow to constraint the segmentation procedure. At the end of the process, each marker must belong to an independent region.

## 10.3.1. _Choice of arc weights for segmentation_

It is necessary to define the weights associated with the arcs or graph edges, which allow us to achieve a good segmentation. The weights of the arcs are defined, so that the optimal structure of the graph enables good object contour detection, which are present in the image, in order to, for example, separate the objects in relation to their contrast [BOY 01b, BOY 03, FAL 04].

Let's take _G_ = ( _V, E, W_ ) a pixel adjacency graph, where _V_ is the set of vertices (nodes), E the set of edges (arcs) and W the set of weights. One way of attributing the weights to the arcs is given by the following equation:

[10.1]

where i and j are two graph vertices, _p i_ and _p j_ are the gray levels of the two neighboring image pixels _i_ and _j_ and _dist_ ( _i, j_ ) is the distance function between the two neighboring pixels (for example, the Euclidean distance).

The coefficients _ω i,j_ act as a measurement of similarity between the neighboring pixels, and can be seen as a local estimation of the image gradient module. _ω i,j_ is a strictly positive and increasing function for all values of | _p i − pj_ |. A low value of the arc weight implies that the pixels _i_ and _j_ have similar gray values; on the other hand, a high value of _ω i,j_ is interpreted as a large difference between the gray levels of the pixels _i_ and _j_. It should be noted that any strictly positive and increasing function as a function of | _p i − pj_ | can be used to produce the same results.

Another choice of function often encountered in the literature is the following:

[10.2]

where _α_ is a scale parameter and _h_ ( _i, j_ ) is the distance between several local attributes of the image. The attribute can be of any type. We can observe that, contrary to the previous function, this function does not bring into play any information related to the distribution of the pixels.

Below we give some examples of distance functions which are given:

1) local distance:

[10.3]

where the function _h_ ( _i, j_ ) expresses the spatial proximity of the pixels _i_ and _j_ ;

2) non-local distance:

[10.4]

where _D_ is a descriptor vector. The function _h_ ( _i, j_ ) expresses the proximity between the local descriptors.

3) combination of local and non-local distances:

[10.5]

The choice of the attribute is important for obtaining an optimal segmentation result. For non-noisy and smoothed images, the gray level information can be very efficient. For textured images, we can, for example, consider an attribute vector from a filter bank, as suggested in [MAL 01]. An efficient way to represent the textures without filter bank, is to use the patch around the pixels as an attribute vector. This idea was introduced for texture synthesis [EFR 01] and in image denoising [BUA 05]. The choice of the equation [10.1] can be used for image segmentation, via a minimum spanning tree or shortest path algorithm. Figure 10.6 illustrates the result of image segmentation via the minimum spanning tree with equation [10.1]. However, the inverse of function (so 1/ _ω i,j_ ) must be used in the case of a segmentation that is based on mimimum flows. By basing ourselves on this definition of the weight, we can segment an image by searching for the minimum spanning tree, and by minimizing the sum of the weights of the arcs. In fact, the minimum spanning tree would incorporate and contain the low weight arcs, which implies that the pixels with similar gray level values are included in the same tree. As a result, the edges between two trees have high value weights, which correspond to pixels that have different gray level values [STA 08, SHI 98, SAR 00, VEK 00].

**Figure 10.6.** _Illustration of image segmentation via the minimum spanning tree. The encircled regions are poorly segmented [STA 08]. These images are colored on the Website of this book: perso.ec-lyon.fr/charles-edmond.bichot/livre_partitionnement_

# 10.4. Image segmentation via maximum flows

In this section, we will introduce the principle of image segmentation based on maximum flows. Then we will use the region adjacency graph, which has emerged from a WT type non-supervised transformation. This over-segmentation is calculated from all the minima of the image gradient module. The result of this stage is similar to the one illustrated in Figure 10.3(b).

Assume:

– this transformation preserves all the image contour information. The use of the region graph in place of the pixel graph does not penalize contours detection procedure. This property allows us to approach the minimal surfaces, and geometric geodesics;

– the use of the region graph, instead of the pixel graph, is not detrimental to the detection of homogeneous regions of an image.

## 10.4.1. _Maximum flows for energy minimization_

Over the last few years, graph partitioning has been very commonly used as a process to minimize the energy functions which are associated with the different problems of image processing, such as stereovision, image segmentation and image restoration [BOY 99, BOY 01a, BOY 03, BOY 01b, LI 04]. The basic principle is that graph partitioning implicitly defines an optimal classification of the pixels of an image. A much used approach is the one of maximum flows, which is based on a

**Figure 10.7.** _Illustration of the labeling procedure_

theorem [FOR 56] that asserts that a maximum flow going from a source to a sink is equal to the minimum capacity of the cut where the cut must be removed from the graph, so that no more flows pass between the source to the sink.

Let's consider a graph _G_ = ( _V, E, W_ ). We will assume that two extra vertices s and t are added to the graph, where these terminal vertices are connected to all the vertices of the graph through a set of arcs _E_ _s_ = { _e s,i, i ∈ V_ } and _E t_ = { _e i,t, i ∈ V_ }. We will denote this new graph created this way: _G s,t_ = ( _V_ ∪ { _s, t_ }, _E_ ∪ _E s_ ∪ _E t, W_ ). Hence _a i_ ∈  + the coefficient of an arc _e s,i ∈ Es, bi_ ∈  + is the coefficient of an edge _e i,t ∈ Et_ and _ω i,j_ ∈  + is the coefficient of an edge _e i,j ∈ E_. It must be noted that we will restrict our study to the case of graphs where it is possible to calculate the minimum partition, for example graphs with positive weights.

We associate with a partition of a graph _C s,t_, which separates the vertices _s_ and _t_ , a labeling function _x i_, such that, for _i ∈ V, x i_ = 1, if a path exists from the vertex _i_ to the vertex _s_ with the removal of the edge _C s,t_, and _x i_ = 0 elsewhere. We will allocate a label from the set 0, 1 for each connected component of the graph, after the removal of the partition _C s,t_, depending on the content of the connected component _s_ or _t_. This procedure is illustrated in Figure 10.7.

The weights of each partition _C s,t_ in the graph _G s,t_ can be defined by:

[10.6]

By separating the constant terms from the other terms:

[10.7]

As a result, the coefficient of a partition in the graph can be written as a function _E_ ( _x_ 1,... , _x n_) ( _n_ = _Card_ ( _V_ )) of the binary variable xi:

[10.8]

where _A_ = ∑ _j∈V_ _b i_, _α i = ai − bi_ \+ ∑ _j∈V_ _ω i,j_ , and _β i,j_ = − _ω i,j_ .

Equation [10.7] of the energy function characterizes the binary variables, and can be minimized by using the graph partitioning algorithms.

THEOREM 10.1 (ENERGY MINIMIZATION VIA MAXIMUM FLOW). _Any function that can be written in the form of a sum dependent on more than two binary variables x i_ ∈ {0; 1}:

[10.9]

_can be accurately minimized by the maximum flows, if and only if_ , A ∈  +, _α i ∈   and βi,j_ ∈  −.

The proof of the theorem is in [KOL 04, FRE 05]. This theorem is referred by many researchers who work on minimization via maximum flows. Kolmogorov _et al_. [KOL 04] has characterized the energy functions which can be expressed as a sum that is dependent on more than three binary variables, and minimized by maximum flows. Freedman _et al_. [FRE 05] has generalized these results for n binary variables.

## 10.4.2. _Minimal geodesics and surfaces_

Minimal surfaces and geodesics are widely used in image segmentation, since they allow us to get good results in various applications. In the literature, we will distinguish between two approaches to solve the segmentation problem: first, geodesic active contours [CAS 97], which are based on the optimization of a Riemannian metric for the calculation of the optimal contour. The second approach, proposed by Boykov [BOY 03], is based on integral geometry, in order to calculate the similar contours, through the representation of an image in the form of a graph and maximum flows. In [STA 08] the authors has proposed a technique to approach the geodesics and minimal surfaces by using over-segmentation and maximum flows. The authors have proposed the use of a WT for the calculation of the curves and the minimum surfaces.

### 10.4.2.1. _The approach of differential geometry_

Differential geometry is the application of differential calculation tools to geometry. This approach is used in many fields: mechanics, physics, etc. The use of this approach in image segmentation was introduced by Kass _et al_. [KAS 88], under the name of "snakes", as well as other models of active contours. The principle of the segmentation methods based on this approach is shown in [SET 99]. This representation allows changes in the topology of the contour curves, and a better manipulation of the numerical schematics to obtain a minimization of the energy. Cassel _et al_. [CAS 97] introduced an approach based on geodesic active contours, which simplifies the energy function to minimize. The problem is expressed in the form of energy minimization:

[10.10]

where _C_ ( _s_ ) = ( _x_ ( _s_ ), _y_ ( _s_ )) is a closed and parameterized curve, s is the curvilinear abscissa, with _s_ ∈ [0, 1]. _g_ is a positive, and strictly decreasing function, and   ∇ _I_ ( _C_ ( _s_ ))   is the image gradient module _I_ on the curve _C_.

This method aims to find a curve, so that the contrast of the image (gradient modulus) on the curve is maximal. This method is equivalent to the minimization of the length of the curve _C_ in relation to a Riemannian metric. In our case, the Riemannian metric can be written in the following form:

[10.11]

where _κ s_ designates the tangent unit vector of the curve _C, M_ is the metric tensor, positive definite matrix, which therefore defines the local matrix. In the case of the geodesic active contours, the local Riemannian metric at a point is given by the following positive symmetric matrix:

[10.12]

where _Id_ is the identity matrix. The length of a geometric curve under a Riemannian metric given by the equation [10.12], is therefore equal to the energy equation [10.10]. The principle of the geodesic active contours is to minimize the energy via a gradient descent.

Geodesic active contours have taken precedence over the snakes, since they are capable of dividing and combining, to allow simultaneous detection of several objects. They can accomplish a more accurate contour detection and are independent of initialization. In addition, extension to 3D cases is possible.

On the other hand, even though there are many advantages of geodesic active contours over snakes are many, certain negative points still remain, in particular the sensitivity to noise in the image. Their main drawback is their non-linearity. Their expression is too strict and does not allow the addition of extra constraints. Also, the initialization of the geodesic active contours is an important and delicate parameter. Due to the gradient descent and a non-convex expression, these geodesic active contours do not converge toward a global optimum. This fact will require us, to therefore provide an initial contour close to the desired solution, which is not always possible. It should be noted that there are convex expressions of these active contours, which give much better results [APP 05, BRE 07]. In addition, in the model of [BRE 07], the regional energies can be considered, by giving a relatively flexible model. In the following section, we will consider the computation of a curve or surface minimizing the energy.

Boykov [BOY 03] proposed a calculation of the minimal geodesic surfaces by using the equation of Cauchy-Crofton. Cauchy has introduced a equation for the calculation of the length of a curve. He has used the concept of absolute projection, in this equation.

Let's consider _L_ ( _r_ , Θ) that emerges from the origin and forms an angle Θ with the X-axis, the absolute projection of a curve _C_ onto _L_ is the quantity:

[10.13]

where _N_ ( _r_ , Θ) is the number of points of the curve, where the orthogonal projection is the point ( _rcos_ Θ, _rsin_ Θ) of _L_. In other words, it is the number of intersections between the line and the curve _C_. The length of a regular curve _C_ in  2 is given by:

[10.14]

This equation can also be extended to the Riemannian space [BOY 03]; in this case, the length of the curve _C_ in relation to a metric tensor _M_ , is given by:

[10.15]

**Figure 10.8.** _Illustration of the differential geometry approach_

where _u L_ are the unit vectors in the direction of the line L. This expression is valid for any differentiable and regular curve in  2.

In order to adapt the previous expression to the case of the image, we must discretize them. Let's take Nn an adjacency system which defines the isotropic grid of size _d_. Nn can be described in the form of a finite set of vectors _e k_ , _N n_ = _e k_ : 1 ≤ _k_ ≤ _n_ , where _n_ is the number of vectors describing the adjacency system. Each vector _e k_ generates a family of lines, where each straight line is separated by a distance Δ _r k_ of the nearest line of the family, as illustrated in Figure 10.8. Now let's consider Θk an angular parameter; for each fixed set Θ _k_ , we obtain a family of parallel lines, separated by a distance Δ _r k_ (Figure 10.8).

The approximation of equation [10.14], in a discrete space, allows us to obtain the following equation, for the calculation of the Euclidean length of a curve _C_ [BOY 03]:

[10.16]

where _i_ is the index of the _k nth_ family of lines, _n c_( _i, k_ ) the number of intersections of the line _i_ , of the _k nth_ family of lines, with the curve _C,n c_( _k_ ) = ∑ _i nc_( _i, k_ ) is the total number of intersections of the _k nth_ family with the curve C. Equation [10.16] is used for the calculation of the length of a curve defined on a discrete grid. In addition, this expression is related to the calculation of the cost of a _maximum flow_.

**Figure 10.9**. _Illustration of the minimum geodesics and surfaces approach_

## 10.4.3. _Minimum geodesics and surfaces via maximum flows_

Boykov [BOY 03] introduced the calculation of minimum geodesics, and surfaces that separates two sets of pixels: the object markers and background markers. Two extra vertices s and t are connected, respectively, to the object markers and the background markers. "s-edges" and "t-edges", along with the edges that are connected to s and t, provide lot of abilities for ensuring that the segment regions contain, respectively, the background markers and the object markers. This is illustrated in Figure 10.9.

Boykov [BOY 03] related the cost of maximum flow to the length of a curve, which crosses the edges of a cut, as we have previously seen. Let's consider an image that is encapsulated in a discrete grid, along with Nn, an adjacency system defined on the image. As we have previously seen, the adjacency system that defines a family of lines. The cost of maximum flow C in the constructed graph is defined by:

[10.17]

where _n c_( _k_ ) is the number of edges of the family _k_ included in the maximum flow and _ω k_ is the weight of the edges of the family _k_.

The Cauchy-Crofton equation given by equation [10.16] can be directly used, to set the values of the coefficients of the graph arcs, in such a way that the cost of the

**Figure 10.10.** _Cell segmentation: three separate segmentations must be carried out, one for each cell. Figure (b), the very square contours are of due to the low isotropic nature of the graph. Figure (c), the contours are better quality, the segmentation is globally optimal. These images are in color on the Website:perso.ec-lyon.fr/charles-edmond.bichot/livre_partitionnement_

maximum flow estimates the Euclidean or Riemannian length of a curve cutting the arcs:

[10.18]

This formulation is important because by using only the trivial edges of the 4- connected network in 2D or 6-connected network in 3D, the segmentation by graph cut show some "metric" artifacts. Figure 10.10 illustrates these artifacts.

It should be noted that equation [10.18] can be extended to the case of a Riemannian metric and the 3D space. Unfortunately, a method that uses a pixel graph cannot be applied to large-sized images, in particular for 3D medical imaging. In [STA 08], the authors propose an approximation of the minimum geodesics and surfaces, to overcome this problem. However, this last approach requires sufficiently clear contours. In the case of blurry or partially non-existant contours, the segmentation obtained is not reliable.

Figure 10.10 shows an example of segmentation which uses maximum flows of a lizard sitting on a rock. The lizard has the same color as the rock, which increases the complexity of the segmentation process. The result shown in Figure 10.10(b) shows that the information of intensity does not allow us to segment the image efficiently (one leg of the lizard is not completely segmented). However, the use of a texture tensor gives a good segmentation result.

**Figure 10.11.** _Example of segmentation result [BOY 01a]_

## 10.4.4. _Continuous maximum flows_

The maximum flows can be placed in the continuous domain [STR 83]. It is, therefore, possible to propose efficient algorithms to calculate the geodesic active contours [APP 05], total variation [CHA 97, NIK 06] or minimum surfaces [APP 06] are also possible. It is, therefore, interesting to discretize these continuous approaches, in order to obtain algorithms with more interesting properties; for example, a proof of convergence of the discrete version of the approach [APP 06] is proposed in [COU 08a]. We can, in many cases, minimize the continuous functional of the geodesic active contours precisely and globally [APP 05]. We can also minimize the continuous interpretation through a minimal surface of the problem in 3D [APP 06]. It must be noted that this approach, as compared to the other models discussed, is very efficient in 3D. In addition, dual expressions have been recently proposed, which have originated from total variation: [CHA 97, KOL 09, NIK 06].

# 10.5. Unification of segmentation methods via graph theory

Recently, Couprie _et al_. [COU 09a] proposed a unification of the expression of the image segmentation problem that uses maximum flows, random walk, shortest path, and watershed by optimal spanning forest. This method is named _power watersheds_. Before presenting the new approach, we will give the principle of segmentation via these different models.

_Random walker (diffusion in the graph)_ : in this approach, the time is discretized _t_ = (0, 1,...). At any moment, a walker is located on a vertex and moves, during the following instant, toward another vertex, where this vertex is chosen randomly and uniformly among the neighboring vertices. The series of vertices visited thus constitutes a random walk. The probability of transition from the vertex i toward the vertex _j_ at each iteration is given by:

Hence the definition of the transition matrix _P_ of the Markov chain corresponds to the random walk. The aim of introducing this transition matrix is the following: the exploration generated by random walk allows us to determine the topological properties of the graph, which are related to the spectral properties of _P_ .

The algorithm proposed by Grady [GRA 07], based on the random walker (RW), is expressed on a weighted graph, and provides the labels for the unmarked vertices, by allocating the label of the marker where the probability of sending a random walker is the maximum. We can also interpret this approach as the allocation of the unmarked pixels to the classes for which there exists a minimum diffusion distance, such as a semi-supervised learning algorithm, or an interactive version of the standardized cuts. In addition, we can say that the algorithms used for the deletion of the image background ( _image matting_ ), by using quadratic minimization via the Laplacian matrix, are based on the same approach of pixel regrouping, but with different strategies, in order to determine the weighting function of the arcs. The diffusion distances allow us to avoid segmentation leaks and reduce trends. However, the segmentation edges can be strongly affected by the placement of the markers in the case of maximum flows.

_(Geodesic) shortest paths_ : the shortest path algorithm allocates each pixel to the foreground, if the shortest path, in relation to the paths which lead to all the background markers, is the one which proceeds from this pixel to the foreground marker. The paths are weighted by the content of the image, in the same way as the maximum flows and the RW approaches. This approach has recently been updated. However variants of this idea appeared at the same time [FAL 04, XUE 07]. The main advantage of this algorithm is its rapidity and ability to avoid local minima. However, the approach is very sensitive to the position of the markers, and is not efficient in the case of 4-connectivity.

Intuitively, the watershed function (considered as a topographical surface) is formed of places a drop of water could flow toward the different minima. The tool that allows the expression and proof of this proposition, involves the usage of optimal spanning forests relative to the minima. In order to adapt WT to marker-based segmentation, the image gradient can be considered as a relief map. In addition, instead of minima, the markers can be placed by the user, or found automatically, to specify image segmentation in the concerned regions. The maximum (minimum) spanning forest (MSF) calculates several trees which can cover all the graph vertices, where each tree is precisely linked to a connected marker. In addition, the weight of the set of trees which are at a maximum (minimum), is also determined. An optimal spanning forest can be calculated with the algorithm proposed by Cousty _et al_. [COU 08a] in quasi-linear time. The WT are greatly used in image segmentation, numerous efficient variants are available, which can be easily implemented. However, the segmentation results suffer from leaks and solution degeneration on the plateaux of the weight function.

A generalized model, to produce a segmentation s, including the maximum flows, shortest path and RW, was proposed by Grady _et al_.\

[10.19]

with _x_ ( _F_ ) = 1, _x_ ( _B_ ) = 0, and  .

The authors have shown that the maximum flows give a good solution to this model, when _q_ = 1, and RW gives a good solution if when _q_ = 2. The approach based on the shortest path (geodesics) can provide a good solution to this model when _q_ → ∞.

The generalization of this model, in order to include segmentation by using WT, is simply done by separating the exponent on the weights and variables. In other words, we introduce the parameter _p_ to define a new segmentation model:

[10.20]

with _x_ ( _F_ ) = 1, _x_ ( _B_ ) = 0, and  .

When _p_ is a "small" finite value, the different values of _q_ can be interpreted respectively as maximum flow ( _q_ = 1), and RW ( _q_ = 2) methods. When _q_ and _p_ tend to infinity with the same speed, then the solution of [10.20] can be calculated by using the shortest path (geodesic) algorithm.

It should also be noted that when q = 1 (maximum flows) and p → ∞, the minimum of equation [10.20] can be calculated by a maximum spanning forest algorithm [ALL 10], which can be seen as a WT [COU 09b]. These four algorithms form the basis of most of the image segmentation methods present in the literature.

**Figure 10.12.** _Illustration of the progressive convergence of power watersheds obtained for q = 2 [COU 09a]. These images are in color on the Website of this book: perso.ec-lyon.fr/charles-edmond.bichot/livre_partitionnement_

By having _p_ → ∞ and by varying the power _q_ , we can obtain a new family, which has not been explored so far [COU 09a], of segmentation models, called _power watersheds_. The authors have provided the proof that, when _p_ → ∞, there exists a value of _p_ beyond which any algorithm can be calculated by using an optimal spanning forest. An algorithm which calculates this limit is proposed in [COU 09a].

An illustration of the progressive convergence of the power watersheds is shown in Figure 10.12. The original image with the markings is in Figure 10.12(a), with an aim to efficiently segment the apple, without touching the grapes. The segmentation is carried out with _q_ constant ( _q_ = 2) and the results obtained, for different values of 8.25 and infinity of _p_ , are shown in Figures 10.12(b), 10.12(c) and 10.12(d), respectively. As we can observe, the result in the last case is a good quality segmentation. This illustrates the efficiency of _power watersheds_.

The value _q_ = 2 has been used, since we know that, for _q_ = 2, the problem described by equation [10.20] has a single minimum.

In order to illustrate the proposed generalization, we will show 5 different combinations of _p_ and _q_ , which correspond to a maximum flow, the random walker (RW), the shortest path, the watershed line / maximum spanning forest and the _power watershed_ , with a parameter _q_ = 2. The images used originate from the _Microsoft Grabcut database_ , where the database is available online [ROT 04], which is composed of 50 images provided with markings. Unfortunately, the markers provided by the _Grabcut_ database are generally equidistant from the actual contour field. In order to eliminate any preference related to the placement of these markers in terms of the comparison results, an extra set of markers has been produced, which significantly weakens the original markers of the object.

Figure 10.13 shows 5 examples of segmentation results. The original marked image is in Figure 10.13(a), where we aim to extract the object from the background. The difficulty in this example is the background color, which is not much different from that of the object. In other words, there is a low contrast between the object and the background, which makes the use of segmentation techniques, such as thresholding, impossible. The segmentation results for the five algorithms are shown in Figures 10.13(b) to 10.13(d).

Finally, we observe that, for _q_ = 2, the _power watersheds_ give a unique global optimum, irrespective of the number of markers (see Figure 10.13); this minimum is obtained, in practice, in quasi-linear time. On the contrary, the maximum flows form an NP-complete problem, as soon as the number of markers exceeds 2.

# 10.6. Conclusions and perspectives

The application of graph theory to image analysis dates back to the 1970s. However, keen interest in the techniques for the application of graph partitioning algorithms has emerged over the last few years. In this chapter, we have defined the operation of segmentation, and have shown that an image, is in fact, only an undirected weighted graph. As a result, most of the graph partitioning techniques can be applied under certain conditions.

The research in this field is very active. The hydridization of graph theory with those of deformable models is a promising research direction, and the initial results are very encouraging. In addition, a new family of segmentation models, called _power watersheds_ , was introduced. A recent article of Grady [GRA 09] reports on a reformulation of the Mumford-Shah segmentation model by using graphs. It will certainly open up a new field of application for graphs in image segmentation. In terms of perspectives, we observe that for the last few years there has been a convergence of filtering methods, deconvolution and segmentation. The communities of signal, image, vision and image analysis processing have each proposed expressions based on energy minimization, which are more and more compatible, as optimization methods become

**Figure 10.13**. _Segmentation examples obtained by different algorithms using weakened markers. These images are in color on the Website of this book: perso.ec-lyon.fr/charles-edmond.bichot/livre_partitionnement_

**Figure 10.14.** _Example of precise multi-label segmentation obtained by the power watersheds method (for q = 2). These images are in color on the Website of this book: perso.ec-lyon.fr/charles-edmond.bichot/livre_partitionnement_

more general and efficient. We can, therefore, estimate that in the near future, a unified approach of these problems will be proposed. We estimate that the discrete methods introduced in this chapter, or those that will be inspired by them, will play a crucial role in their resolutions.

# 10.7. Bibliography

[ADA 94] ADAMS R., BISCHOF L., "Seeded region growing", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 16, no. 6, p. 641-647, 1994.

[ALL 10] ALLÈNE C., AUDIBERT J.-Y., COUPRIE M., KERIVEN R., "Some links between extremum spanning forests, watersheds and min-cuts", _Image and Vision Computing_ , vol. 28, no. 10, October 2010.

[APP 05] APPLETON B., TALBOT H., "Globally optimal geodesic active contours", _Journal of Mathematical Imaging and Vision_ , vol. 23, no. 1, p. 67-86, 2005.

[APP 06] APPLETON B., TALBOT H., "Globally minimal surfaces by continuous maximal flows", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, p. 106-118, 2006.

[BEA 06] BEARE R., "A locally constrained watershed transform", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 28, no. 7, p. 1063-1074, 2006.

[BEU 93] BEUCHER S., MEYER F., "The morphological approach to segmentation: the watershed transform," in DOUGHERTY E.R. (ed.), _Mathematical Morphology in Image Processing_ , vol. 12, Marcel Dekker, New York, p. 433-481, 1993.

[BOY 99] BOYKOV Y., VEKSLER O., ZABIH R., "Markov random fields with efficient approximations", _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , IEEE, p. 648-655, 1999.

[BOY 01a] BOYKOV Y., JOLLY M., "Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images", _Proceedings of the 8th International Conference on Computer Vision_ , IEEE, p. 105-112, 2001.

[BOY 01b] BOYKOV Y., VEKSLER O., ZABIH R., "Fast approximate energy minimization via graph cuts", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 23, no. 11, p. 1222-1239, 2001.

[BOY 03] BOYKOVY., Kolmogorov V., "Computing geodesics and minimal surfaces via graph cuts", _Proceedings of the 9th International Conference on Computer Vision_ , IEEE, p. 26-33, 2003.

[BRE 07] BRESSON X., ESEDOGLU S., VANDERGHEYNST P., THIRAN J.-P., OSHER S., "Fast global minimization of the active contour/snake model", _Journal of Mathematical Imaging and Vision_ , vol. 28, no. 2, p. 151-167, 2007.

[BUA 05] BUADES A., COLL B., MOREL J.M., "A review of image denoising algorithms, with a new one", _Multiscale Modeling and Simulation_ , vol. 4, no. 2, p. 490-530, 2005.

[CAS 97] CASELLES V., KIMMEL R., SAPIRO G., "Geodesic active contours", _International Journal of Computer Vision_ , vol. 22, no. 1, p. 694-699, 1997.

[CHA 97] CHAMBOLLE A., LIONS P.-L., "Image recovery via total variation minimization and related problems", _Numerische Mathematik_ , vol. 76, no. 2, p. 167-188, 1997.

[COU 08a] COUPRIE C., GRADY L., TALBOT H., NAJMAN L., DCMF: discretized continuous max flows, Report no. IGM 2008-07, Institut Gaspard Monge, 2008.

[COU 08b] COUSTY J., BERTRAND G., COUPRIE M., NAJMAN L., "Fusion graphs: merging properties and watersheds", _Journal of Mathematical Imaging and Vision_ , vol. 30, no. 1, p. 87-104, 2008.

[COU 08c] COUSTY J., BERTRAND G., NAJMAN L., COUPRIE M., "Weighted fusion graphs: merging properties and watersheds", _Discrete Applied Mathematics_ , vol. 156, no. 15, p. 3011- 3027, 2008.

[COU 09a] COUPRIE C., GRADY L., NAJMAN L., TALBOT H., "Power watersheds: a new image segmentation framework extending graph cuts, random walker and optimal spanning forest", _Proceedings of the 12th International Conference on Computer Vision_ , IEEE, p. 731-738, 2009.

[COU 09b] COUSTY J., BERTRAND G., NAJMAN L., COUPRIE M., "Watershed cuts: minimum spanning forests and the drop of water principle", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 31, no. 8, p. 1362-1374, 2009.

[EFR 01] EFROS A.A., FREEMAN W.T., "Image quilting for texture synthesis and transfer", SIGGRAPH '01: _Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques_ , Hermès, p. 341-346, 2001.

[FAL 04] FALC AO A.X., STOLFI J., DE ALENCAR Lotufo R., "The image foresting transform: theory, algorithms, and applications", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 26, no. 1, p. 19-29, 2004.

[FOR 56] FORD L.R., FULKERSON D.R., "Maximal flow through a network", _Canadian Journal of Mathematics_ , vol. 8, p. 399-404, 1956.

[FRE 05] FREEDMAN D., DRINEAS P., "Energy minimization via graph cuts: settling what is possible", _CVPR '05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , IEEE, p. 939-946, 2005.

[GRA 07] GRADY L., SINOP A.K., "A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm", _Proceedings of the 11th International Conference on Computer Vision_ , IEEE, 2007.

[GRA 09] GRADY L., ALVINO C., "The piecewise smooth Mumford-Shah functional on an arbitrary graph", IEEE Image Processing, vol. 18, no. 11, p. 2547-2561, 2009.

[KAS 88] KASS M., WITKIN K., TERZOPOULOS D., "Snakes: active contour models", _International Journal of Computer Vision_ , vol. 1, no. 4, p. 321-331, 1988.

[KOL 04] KOLMOGOROV V., ZABIH R., "What energy functions can be minimized via graph cuts?", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 26, no. 2, p. 147-159, 2004.

[KOL 09] KOLEV K., KLODT M., BROX T., CREMERS D., "Continuous global optimization in multiview 3D reconstruction", _International Journal of Computer Vision_ , vol. 84, no. 1, p. 80-96, 2009.

[LI 04] LI Y., SUN J., TANG C.-K., SHUM H.-Y., "Lazy snapping", _ACM Transactions on Graphics_ , vol. 23, no. 3, p. 303-308, 2004.

[MAL 01] MALIK J., BELONGIE S., LEUNG T., SHI J., "Contour and texture analysis for image segmentation", _International Journal of Computer Vision_ , vol. 43, no. 1, p. 7-27, 2001.

[MCQ 67] MCQUEEN J., "Some methods for classification and analysis of multivariate observations", _Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability_ , University of California Press, p. 281-297, 1967.

[MEY 02] MEYER F., VACHIER C., "Image segmentation based on viscous flooding simulation", in Talbot H., Beare R. (eds), _Proceedings of the 6th International Symposium on Mathematical Morphology_ , CSIRO Publishing, p. 69-77, 2002.

[MEY 10] MEYER F., NAJMAN L., "Segmentation, minimum spanning tree and hierarchies", in NAJMAN L., TALBOT H. (eds), _Mathematical Morphology_ , ISTE Ltd London, John Wiley & Sons, New York, 2010.

[NAJ 96] NAJMAN L., SCHMITT M., "Geodesic saliency of watershed contours and hierarchical segmentation", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 18, no. 12, p. 1163-1173, 1996.

[NAK 09] NAKIB A., OULHADJ H., SIARRY P., "Fractional differentiation and non-pareto multiobjective optimization for image thresholding", _Engineering Applications of Artificial Intelligence_ , vol. 22, no. 2, p. 236-249, 2009.

[NIK 06] NIKOLOVA M., ESEDOGLU S., CHAN T., "Algorithms for finding global minimizers of image segmentation and denoising models", _SIAM Journal on Applied Mathematics_ , vol. 66, no. 5, p. 1632-1648, 2006.

[ROS 70] ROSENFELD A., "Connectivity in digital pictures", _Journal of the Association for Computing Machinery_ , vol. 17, no. 1, p. 146-160, 1970.

[ROS 82] ROSENFELD A., KAK A., _Digital Picture Processing_ , Academic Press, 1982.

[ROS 83] ROSENFELD A., "On connectivity properties of gray scale pictures", _Pattern Recognition_ , vol. 16, no. 1, p. 47-50, 1983.

[ROT 04] ROTHER C., KOLMOGOROV V., BLAKE A., "Grabcut: interactive foreground extraction using iterated graph cuts", _Proceedings of SIGGRAPH, ACM Transactions on Graphics_ , ACM, vol. 23, no. 3, p. 309-314, 2004.

[SAR 00] SARKAR S., SOUNDARARAJAN P., "Supervised learning of large perceptual organization: graph spectral partitioning and learning automata", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 22, no. 5, p. 504-525, 2000.

[SET 99] SETHIAN J., "Level set methods and fast marching methods: evolving interfaces in computational geometry", _Fluid Mechanics, Computer Vision and Materials Science_ , Cambridge University Press, 1999.

[SHI 98] SHI J., BELONGIE S., LEUNG T., MALIK J., "Image and video segmentation: the normalized cut framework", _Proceedings of IEEE International Conference on Image Processing (ICIP)_ , IEEE, p. 943-947, 1998

[STA 08] STAWIASKI J., Morphologie mathématique et graphes : application à la segmentation interactive d'images médicales, PhD Thesis, Ecole Nationale Supérieure des Mines de Paris, 2008.

[STR 83] STRANG G., "Maximal flow through a domain", _Mathematical Programming_ , vol. 26, p. 123-143, 1983.

[VEK 00] VEKSLER O., "Image segmentation by nested cuts", in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 13-15 June 2000, vol. 1, Hilton Head Island, SC, USA, p. 339-344, 2000.

[VIN 91] VINCENT L., SOILLE P., "Watersheds in digital spaces: an efficient algorithm based on immersion", _IEEE Transaction on Pattern Analysis and Machine Intelligence_ , vol. 13, no. 6, p. 583-598, 1991.

[XUE 07] XUE B., SAPIRO G., "A geodesic framework for fast interactive image and video segmentation and matting", _IEEE 11th International Conference on Computer Vision_ , IEEE, p. 1-18, http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4408931, 2007.

* * *

1 Chapter written by Amir Nakib, Laurent Najman, Hugues Talbot and Patrick Siarry.

# Chapter 11

# Distances in Graph Partitioning 1

# 11.1. Introduction

The study of protein-protein interaction networks, which we have undertaken over the past few years [BRU 03, BRU 04, BAU 06, BEC 09], led us to carry out a comparative study of graph partitioning methods [GUÉ 05, GUÉ 08]. This study is related to undirected simple graphs with weighted or unweighted edges. Our aim is to construct, in these sparse biological graphs, clusters in which the edge density is high, in comparison with the graph as a whole, in other words clusters that are _communities_. One of the conclusions of these works, in the field of computer science, is that the most common clustering methods (k-means), applied to the graph distances, give excellent results, in comparison with more recent algorithms. We will cover this approach here, focusing uniquely on the distance methods in graph partitioning in order to determine the best adapted distance formula.

Graph distances were studied in depth by Kuntz in her thesis [KUN 92]. Her work was related to the representation of electronic circuits (VSLI). These large graphs cannot be drawn on a single page and have to be cut up. This problem, is of course, related to the minimum cut, so as to mask a minimum number of edges. In contrast, Kuntz attempted to study the distances where the distances allowed a Euclidean representation, which led her to compare _local_ dissimilarity indices. These indices between two vertices _x_ and _y_ , can be calculated from the adjacency lists of _x_ and _y_. They have been introduced in ecology for the comparison of binary data and are therefore applicable to graphs. After her mathematical study of these indices, it appears that the Czekanowski-Dice distance [DIC 45] is very well adapted for graph clustering.

Numerous studies on graph partitioning have been conducted by many authors, from various backgrounds. They have proposed constructive methods of dense clusters: [BAT 99, BAR 00, MOO 01, ENR 02, GIR 02, ROU 03, BAD 03, BRU 04, NEW 04, DUT 05, BRO 06]. These algorithms are often based on the weighting of edges or their presence/absence. But these weighting algorithms are not similarity indices defined on any pair of vertices. Even if it is possible to transform them into distances, we would not perform this task, on the one hand due to respect of the authors who did not do it, but maybe tried to, and on the other hand since this would lead us to very lengthy developments. However, in the last few years a very natural distance has appeared (2006); it is, based on random walks in graphs. This distance, which we name after its authors, Pons-Latapy distance, is very easy to calculate and is worth being compared to the Czekanowski-Dice distance, on the condition that it is extended to weighted graphs, which is what we did first and foremost [ANG 08].

We adopt the following standard notations. Let _X_ be a set of _n_ vertices, _E_ the set of _m_ edges, and Γ = ( _X, E_ ) the corresponding graph. We will admit that it is connected and, if not, its different connected components will be processed separately. For every subset _Y_ of _X_ , let Γ( _Y_ ) be the set of vertices outside of _Y_ , which are adjacent to _Y_ ,

Γ( _Y_ ) = { _x_ ∈ _X_ \ _Y_ such that ∃ _y_ ∈ _Y_ , ( _x, y_ ) ∈ _E_ }

and  ( _Y_ ) = _Y_ ∪ Γ( _Y_ ). The neighborhood of _x_ is therefore Γ( _x_ ). The degree of _x_ x is noted _Dg_ ( _x_ ) = |Γ( _x_ )|, and _δ_ designates the maximum degree of the graph. Let _E_ ( _Y_ ) be the set of internal edges of the class _Y_ ⊂ _X_ :

_E_ ( _Y_ ) = {( _x, y_ ) ∈ _E_ such as _x_ ∈ _Y_ and _y_ ∈ _Y_ }

# 11.2. The Dice distance

We have used the Czekanowski-Dice distance [DIC 45] for protein-protein interaction graphs (or _PPI_ graphs) to construct classes with the help of several methods PRODISTIN [BRU 03, BAU 06], CLASDENS [GUÉ 04, BRU 04], GRAPH PART [GUÉ 05]. The aim was to associate cellular functions with these classes, and the results which we obtained, as well as the display of the subgraphs generated by the classes, proved the efficiency of this proximity measurement. In fact, the Czekanowski-Dice distance is not a _true_ distance but a dissimilarity index; two adjacent vertices which only have common adjacent vertices are at a distance 0, and this index does not fulfill the triangular inequality [FIC 84]. Nevertheless we will continue (like everyone else) to call it a distance, denoted _D_ Γ or simply _D_ when there is no ambiguity.

Each value _D_ ( _x, y_ ), which only takes into account the lists of adjacent vertices to _x_ and _y_ , is defined by:

[11.1]

where Δ is the symmetric difference between two sets. We observe that   is used, which correspond to the addition of a loop to each vertex, and so _x_ becomes adjacent to itself. We use this distance for several reasons:

– it has proven to be very efficient in graph partitioning methods [KUN 92, GUÉ 05], more than the number of edges of the shortest path between _x_ and _y_ , which we will verify;

– it is a _local_ measurement, since _D_ ( _x, y_ ) is computable from the only vertices, adjacent to _x_ or y;

– every pair of vertices separated by more than two edges has a maximum distance equal to 1, so there are less than _n_ ( _n_ − 1)/2 values to calculate;

– due to this fact, this distance is computable with _O_ ( _nδ_ 3) in the worst case and, in practice, _δ_ is replaced by the mean degree which, in the PPI graphs, is very low (around 6) compared to _n_.

The value _D_ ( _x, y_ ) is the quotient of two quantities, namely the _specific part_ to _x_ and to _y_ , written _P spe_( _x, y_ ), and formed by the vertices, which are only adjacent to one of the two, and the _total part_ , denoted as _P tot_( _x, y_ ), which also contains common adjacent vertices. If _x_ and _y_ are not connected by an edge, these two vertices are counted in both parts. However, if they are adjacent, they only come into play in the total part. Therefore, this distance gives different values to _D_ ( _x, z_ ) (and _D_ ( _y, z_ )) depending on whether _x_ and _y_ are adjacent or not. We also observe that the intersection Γ( _x_ ) ∪ Γ( _y_ ) is counted twice in the total part.

EXAMPLE.– Let's consider the graphs of Figure 11.1 which only differ by the edge ( _x, y_ ). We measure the two distances between vertices, depending on whether the edge ( _x, y_ ) is in the graph or not. The results are given in Table 11.1.

If ( _x, y_ ) is absent,  ( _x_ ) = { _x, z, t, u_ }, and  ( _y_ ) = { _y, z, t, v, w_ }. We have _P spe_( _x, y_ ) = { _x, y, u, v, w_ } and _D_ ( _x, y_ ) = 5/9 = 0.556. If ( _x, y_ ) is present,  ( _x_ ) = { _x, y, z, t, u_ } and  ( _y_ ) = { _y, x, z, t, v, w_ }. We have _P spe_( _x, y_ ) = { _u, v, w_ } and D( _x, y_ ) = 3/11 = 0.273. We observe that all the values related to _x_ or _y_ are different.

**Figure 11.1.** _Two simple unweighted graphs_ Γ+ _and_ Γ−

**Table 11.1.** _Czekanowski-Dice distance values; the case where_ ( _x, y_ ) ∈ _E is in the lower left part, and the case where_ ( _x, y_ ) ∉ _E in the upper right part. The diagonal values, which are equal to 0, are not indicated_

## 11.2.1. _Two extensions to weighted graphs_

Following the evolution of biological data, we have extended the Czekanowski- Dice distance to weighted graphs [ANG 08]. In PPI graphs, an edge ( _x, y_ ) means that the proteins _x_ and _y_ have a direct interaction, meaning that they are in contact at a given moment, and not just that they belong to a _protein complex_ , that is set of proteins carrying out a particular cellular function. This information can be obtained through various ways, in particular with the help of large scale experiments, such as two-hybrid screening, in which the interactions are detected with some uncertainty. In addition, interactions can be corroborated by the scientific literature, so that they can be considered as reliable. Thus, the weight attributed to an edge can be seen as a probability associated with this interaction.

However, another interpretation of the weights is possible; this can be a measurement of the intensity of the interaction, depending on whether it is necessary for performing the function or whether it is secondary, meaning that it makes it easier or accelerates it. This is also a common interpretation in other domains of application, such as economics, where weighting can quantify transaction volumes between parties, or sociology where the weights can mark the intensity or length of an association or partnership.

Consequently, we propose two equation adapted to these interpretations. The first corresponds to the case, where the weightings are _probabilities_ , and the second to the case where they are _intensities_. These are still extensions of the Czekanowski-Dice equation since, if the weights are all equal to 1, they again give the values of the unweighted distance. In both cases, the weight function, _w_ : _X_ × _X_ → [0, 1], is as large as the interaction is probable or strong. The value 0 corresponds to the absence of edge, and value 1 to a reliable information or an essential contact.

### 11.2.1.1. _The weightings are probabilities_

If the weightings are probabilities, the distance denoted _D p_ between every pair ( _x, y_ ) is related to the probability of the potential edge ( _x, y_ ). Hence, _D p_( _x, y_ ) is a weighted sum of the distances corresponding to the Γ− configurations, in which the edge (x,y) does not exist (with a weight 0), and Γ+ in which it definitely exists (weight 1):

[11.2]

However, the configurations Γ+ and Γ− are themselves weighted by the probabilities of the edges of _E_ ( _Y_ ), with _Y_ = Γ( _x_ ) ∪ Γ( _y_ ). The computation of _Dp_ ( _x, y_ ) should be done by summing over the subsets of _Y_ , each one being weighted by the product of the edge weights present in this subset. For complexity reasons (enumerating subsets of a set), we have defined a simpler equation.

1) in the case of Γ− the part specific to _x_ and to _y_ is composed of:

– edge weights corresponding to the vertices exclusively adjacent to _x_ or to _y_ ;

– for a vertex s connected to both _x_ and y, the difference | _w_ ( _x, s_ ) − _w_ ( _y, s_ )|, which corresponds to situations where s is connected to only one of the vertices;

– two loops on _x_ and _y_ , implicitly of weight 1, in order to remain compatible with the Czekanowski-Dice equation.

Therefore, the specific part weight is equal to | _P spe_( _x, y_ )| = 2 + ∑ _s_ ∈ _Y_ | _w_ ( _x, s_ ) − _w_ ( _y, s_ )|. For the total part, we must count | _P tot_( _x, y_ )| = 2 − 2 × _w_ ( _x, y_ ) + ∑ _s_ ∈ _Y_ _w_ ( _x, s_ ) + _w_ ( _y, s_ ), where the value 2 corresponds to the loops and 2 × _w_ ( _x, y_ ) to the edge ( _x, y_ ) which must be removed since it is absent from the graph Γ−.

∑ 2) in the case of Γ+ the specific part to _x_ and to _y_ is equal to | _P spe_( _x, y_ )| = ∑ _s_ ∈ _Y_ | _w_ ( _x, s_ )− _w_ ( _y, s_ )|; the loops are no longer counted, since _x_ and _y_ are connected. For the total part | _P tot_( _x, y_ )| = ∑ _s_ ∈ _Y_ _w_ ( _x, s_ ) + _w_ ( _y, s_ ) + 2 + 2 × (1 − _w_ ( _x, y_ )), the last term corresponding to the edge ( _x, y_ ) of weight 1 for this configuration.

To come back to formula [11.2], denoting _S_ ( _x, y_ ) = ∑ _s_ ∈ _Y_ | _w_ ( _x, s_ )− _w_ ( _y, s)_ | and _T_ ( _x, y_ ) = ∑ _s_ ∈ _Y_ _w_ ( _x, s_ ) + _w_ ( _y, s_ ) to obtain the distance weighted by probabilities:

[11.3]

EXAMPLE.– Let's calculate the values of _D p_ on the graphs of Figure 11.2 with the same conventions, Γ+ in the lower left part and Γ− in the upper right part. Table 11.2 shows the resulting values.

**Figure 11.2.** _Graphs_ Γ+ _and_ Γ− _whose the edges are weighted_

**Table 11.2.** _Distances weighted by probabilities on the graphs_ Γ+ _and_ Γ− _ofFigure 11.2_

If ( _x, y_ ) is absent, _P spe_( _x, y_ ) = 2 + _w_ ( _u, x_ ) + _w_ ( _y, v_ ) + _w_ ( _y, w_ ) + | _w_ ( _x, z_ ) − _w_ ( _y, z_ )|+| _w_ ( _x, t_ )− _w_ ( _y, t_ )|, _P tot_( _x, y_ ) = 2+ _w_ ( _x, u_ )+ _w_ ( _x, z_ )+ _w_ ( _x, t_ )+ _w_ ( _y, v_ )+ _w_ ( _y, w_ ) + _w_ ( _y, z_ ) + _w_ ( _y, t_ ) and _D p_( _x, y_ ) = 4.5/6.9 . If ( _x, y_ ) is present, Γ−( _x, y_ ) = 4.5/6.9 as before and for Γ+( _x, y_ ), we have _P spe_( _x, y_ ) = _w_ ( _u, x_ ) + _w_ ( _y, v_ ) + _w_ ( _y, w_ ) + | _w_ ( _x, z_ )− _w_ ( _y, z_ )|+| _w_ ( _x, t_ )− _w_ ( _y, t_ )|, _P tot_( _x, y_ ) = 4+ _w_ ( _x, u_ )+ _w_ ( _x, z_ )+ _w_ ( _x, t_ )+ _w_ ( _y, v_ ) + _w_ ( _y, w_ ) + _w_ ( _y, z_ ) + _w_ ( _y, t_ ) and Γ+( _x, y_ ) = 2.5/8.9 ; therefore _D p_( _x, y_ ) = 0.4 × 4.5/6.9 \+ 0.6 × 2.5/8.9 = .429.

### 11.2.1.2. _The weightings are intensities_

If the weightings are considered as intensities, the interactions specific to _x_ or to _y_ tend to separate them and the common interactions, to bring them closer. Thus, the links between _x_ and _y_ can be placed into two categories:

– the attractive forces corresponding to the edge ( _x, y_ ) if it exists, and to every vertex s adjacent to both _x_ and y;

– repulsive forces corresponding to vertices attached to only _x_ or y, as well as the loops, in the case where the vertices _x_ and _y_ are not adjacent.

**Table 11.3.** _Distances weighted by intensities on the graphs_ Γ+ _and_ Γ− _ofFigure 11.2_

Let _R_ ( _x, y_ ) be the repulsive part and _A_ ( _x, y_ ) the attractive part corresponding to pair ( _x, y_ ). We get:

[11.4]

[11.5]

We observe that if ( _x, y_ ) ∈ _E_ , the weight of the edge ( _x, y_ ) is counted twice in _A_ ( _x, y_ ).

The distance _D i_, based on the interaction intensities, is defined by:

[11.6]

We observe that if the vertices _x_ and _y_ are separated by more than two edges there is no attractive strength, the first equation is always applied, and the distance is equal to 1, irrespective of the number of edges.

EXAMPLE.– Table 11.3 shows the _D i_ values on the graphs in Figure 11.2.

For the case ( _x, y_ ) ∉ _E_ , _R_ ( _x, y_ ) = _w_ ( _u, x_ ) + _w_ ( _v, y_ ) + _w_ ( _w, y_ ), _A_ ( _x, y_ ) = _w_ ( _x, z_ ) + _w_ ( _x, t_ ) + _w_ ( _y, z_ ) + _w_ ( _y, t_ ) and _D i_( _x, y_ ) = 4/6.9 = 0.580. In the case ( _x, y_ ) ∈ _E_ , _R_ ( _x, y_ ) is interchanged and _A_ ( _x, y_ ) = _w_ ( _x, z_ ) + _w_ ( _x, t_ ) + _w_ ( _y, z_ ) + _w_ ( _y, t_ ) + 2 _w_ ( _x, y_ ) and _D i_( _x, y_ ) = 2/8.1 = 0.247.

# 11.3. Pons-Latapy distance

As far as we know, the Pons-Latapy distance was introduced by these authors in the excellent PhD of Pons [PON 07] and in a common article in 2006. However it may be that other authors could have used it, since it concerns a truncation of a classic random walk approach in graphs.

The idea of using a random walk in a graph, to find clusters, dates back to Van Dongen [DON 00]. It is based on the following principle: at each step, from a vertex x, a connected vertex in   is randomly chosen. If this vertex is in a high density zone of edges, the random walk will tend to stay there, and these zones form basins of attraction; the probability of coming out after a large number of steps is very low. Since the next vertex is randomly chosen, according to a probability distribution having sum equal to 1, this walk can be described as a Markov process, which converges toward a stationary process. In his thesis, van Dongen deduced a graph partitioning algorithm, MCL (for _Markov Chain Clustering_ ) which, for sufficiently contrasted graphs (clear density difference between intra-class and inter-class pairs), gives satisfactory results. However, in sparse graphs, it tends to isolate cliques, thus forming a very large number of small clusters.

By following this principle, Pons and Latapy had conceived the idea of truncating these random walks after a small number of steps, and consider for each vertex _x_ the vector Π _k_ ( _x_ ) where each component π _k_ ( _x, y_ ) is the probability to reach vertex _y_ from _x_ after a _k_ steps of random walk. These probabilities are easily calculated from the adjacency matrix _A_.

For homogenize notations, we admit that the adjacency matrix corresponds to the weights: _A_ ( _x, y_ ) = _w_ ( _x, y_ ). We begin by adding a loop to each vertex _x_ , having weight equal to the mean of the adjacent vertex weights:   and we redefine the degree _Dg_ ( _x_ ) of a vertex _x_ as the sum of the weights of the edges incident to _x_ , meaning the simple line sum of _A_. For unweighted graphs, the adjacency matrix is the classic matrix completed by 1's on its diagonal. Then we construct the

[11.7]

The probabilities of going from one vertex to another by a _k_ -steps walk are given by the _k_ -th power of the matrix Π.

EXAMPLE.– Tables 11.4 and 11.5 show the transition matrices respectively for Figures 11.1 and 11.2.

It is clear that two vertices that are in same zone, will have similar vectors, and Pons and Latapy propose to measure the distance between _x_ and _y_ as the weighted Euclidean distance between the row vectors of _x_ and _y_ in the matrix Π _k_ :

[11.8]

**Table 11.4.** _Transition matrix of the graph_ Γ− _ofFigure 11.1_

**Table 11.5.** _Transition matrix of the weighted graph_ Γ+ _ofFigure 11.2_

since the high degree vertices will tend to attract the random walks and the difference between the components π _k_ ( _x, z_ ) and π _k_ ( _y, z_ ) is normalized by _Dg_ ( _z_ ).

In order to calculate this distance, a parameter has to be set; it is the number of steps to be taken during random walks. The optimal value depends on the graph and the size of the expected clusters. The authors recommend values from 3 to 5, but we will see that much higher values can be more efficient.

# 11.4. A partitioning method for distance arrays

Once the distances between the vertices of the graph are calculated, we consider applying a partitioning algorithm through the optimization of a criterion. Let  _p_ be the set of partitions of _X_ in _p_ clusters. The optimization of a criterion on  _p_ , with very few exceptions, is a problem NP-hard [HAN 97]. The optimization problems for which there exists polynomial complexity partitioning algorithms are: to maximize the smallest inter-cluster distance, no matter how many clusters, and to minimize the diameter of the partition or the sum of the diameters of the clusters, only in case of partitions in only two clusters. Nearly all the other criteria, which have a practical meaning in clustering, are "untreatable", so that we resort to heuristics.

The "k-means" method, designed for partitioning in Euclidean spaces, or at least spaces generated by quantitative variables, in which the notion of mean makes sense, is the most popular method. At iteration, this method calculates a center for each clusters, then assigns each element to its nearest center. The iterations stop when the clusters are stable. If the centers are "centers of gravity", whose the coordinates are the means of the variable values over the cluster, each iteration minimizes an inertia function equal to the sum, for each element, of the square of its distance to the center of gravity of its cluster. The global minimization of the inertia function is not guaranteed.

Considerable progress has been made in this method with the stochastic optimization ideas, which, from a given configuration, analyzes close configurations, more or less randomly selected in its neighborhood, to detect configurations that can improve the chosen criterion. In partitioning, due to the efficiency and simplicity of implementation, _Tabu_ search heuristics are commonly used [GLO 97]. This is simply a transfer of an element from one class to another, to go from a partition _P_ to another partition _P_ ′, and recalculate the criterion. If it is improved so much the better, if not, we can nevertheless accept this transformation since, from _P_ ′, we can gradually change to an optimal partition.

A long time usage of these types of optimization and simulation protocols to compare different criteria [GUÉ 03] has led us to choose an inertia criterion  , based on a distance, without the elements being described by variables. This is a simple extension of the inertia of "k-means", to the case of distance arrays. Without having a center of gravity, it is simply enough to observe that, what matters is the distance between an element _x_ and the center of gravity of its cluster. In the absence of coordinates, this length value can be replaced by the mean distance of _x_ to the elements of its cluster.

Let _P_ be a partition of _X_ in _p_ clusters in which _P_ ( _x_ ) the cluster containing _P_ x. The contribution to the inertia of the element _x_ is proportional to the square of ∑ _y_ ∈ _P_ ( _x_ ) _D_ ( _x, y_ ), and we define the inertia of the partition _P_ as the sum over _X_ of the contributions of each element:

[11.9]

Remark that it is | _P_ ( _x_ )| and not | _P_ ( _x_ )| − 1 which appears in the denominator, and this is for two reasons:

– to remain computable in the case of clusters reduced to a single element (the contribution of a singleton is zero);

– for clusters with two elements _z_ , _t_ where the inertia is better evaluated by _D_ ( _z, t_ )2 rather than 2 _D_ ( _z, t_ )2.

We attempt to minimize   over  _p_. After having tested the "k-means" type of methods, in which the center of gravity of each cluster is replaced by its median, we have abandoned all assigning re-centering procedures, to keep only the indispensable stochastic optimization step. The retained procedure is a simple _Tabu_ search heuristic, initialized by a random partition into p classes. The latter is chosen after several trials in which the class number of each element is uniformly selected at random from {1, . . . , _p_ }, giving more or less balanced partitions. The criterion is calculated for each one and it is the partition _P_ ∗, with the smallest inertia value, which is kept as initial partition.

During each iteration, for each element which is not a singleton, we calculate the variation of the criterion resulting from its transfer to another random cluster, without looking for the one which is best suited; this variation can be calculated in _O_ ( _n_ ). The moved element is the one which minimizes this variation, even if it is positive. Then it remains frozen in its class over several iterations; any movement of this element becomes _tabu_. The length of the tabu list is set to 5 in our simulations. Each iteration requires _O_ ( _n_ 2) operations. After _N bIter_ consecutive trials, where   does not decrease the procedure stops. The best partition found _P_ ∗, is the final partition.

* * *

**Algorithm 11.1.** PartOpt algorithm

* * *

1: **procedure** PartOpt

2: **for** _N bT rials_ trials **do**

3: randomly select a partition _P_ with _p_ clusters;

4: if (  ( _P_ ) <  ( _P_ ∗)) _P_ ∗ = _P_ ;

5: **end for**

6: _P_ = _P_ ∗; _iter_ = 0;

7: **repeat**

8: **for all** _z_ ∈ _X_ **do**

9: randomly select a cluster _C_ ( _z_ );

10: calculate the variation Δ( _z_ ) of the criterion   if _z_ was assigned to _C_ ( _z_ )

11: **end for**

12: Let _x_ be the element which minimizes Δ( _z_ );

13: assign _x_ to _C_ ( _x_ ) (even if Δ( _x_ ) > 0);

14: **if** Δ( _x_ ) < 0 **then** _P_ ∗ = _P_ ; _iter_ = 0;

15: **else** _iter_ \+ +;

16: end if

17: **until** _iter_ < _N bIter_

18: The final partition _P_ ∗ is obtained during the last improvement of  .

19: **end procedure**

* * *

Therefore, 2 parameters are set:

– _N bT rials_ , the number of trials to construct the initial partition;

– _N bIter_ , the maximum number of consecutive unsuccessful iterations (which do not improve the criterion), to end the procedure.

In our simulations, the first is set at _p_ and the second at _n_. Evidently higher values for these parameters can only improve the results.

# 11.5. A simulation protocol

To evaluate the quality of the computed partitions, we return to a simulation procedure previously developed for the comparison of partitioning methods [GUÉ 05], but with other quality criteria.

1) We construct a graph Γ from a random partition _P_ of _X_ into p clusters, used as a model. The edges are randomly selected with two different probabilities, depending on whether the two ends are in the same cluster or not;

2) the _PartOpt_ method generates a partition _Q_ , according to the used distance;

3) the partitions _P_ and _Q_ are compared by quantifying several points of view, the distance between _P_ and _Q_ , or the adequacy of _Q_ to the initial model.

## 11.5.1. _A random graph generator_

The graphs in which we measure the performance of the distances have "natural" clusters of vertices. They are produced, according to the Erdo˝s-Renyi multi-class model, by random selection:

– a partition of _X_ into _p_ clusters, the cluster number of each element being uniformly selected at random, as in the above algorithm;

– edges with predefined probabilities depending on whether the two ends are in a same cluster (internal edge) or in different clusters (external edge). Thus there are two parameters:

\- _p i_: probability of the internal edges,

\- _p e_: probability of the external edges.

For each pair of elements, we randomly select a real number between 0 and 1, and add the corresponding edge if and only if this number is lower than _p i_ (respectively _p e_) when both elements are in identical (respectively different) clusters.

The gap between _p i_ and _p e_ makes the problem of recovering partition _P_ more or less difficult. It is also possible to get a partition _Q_ more adapted to the graph than partition _P_ , when the edge selection has poorly filled the clusters.

## 11.5.2. _Quality of the computed partition_

The comparison of partitions is a central topic in clustering, whether it is to compare partitioning algorithms or to cluster nominal variables, which are themselves partitions. The literature is full of indices defined to compare two partitions _P_ and _Q_ on the same set _X_. Here we use three criteria:

– the first is the _Rand index corrected by chance_ , due to Hubert and Arabie [HUB 85];

– the second is a natural distance between partitions defined by Régnier [RÉG 83], highlighted by Day [DAY 81] and more recently by Charon _et al_. [CHA 05];

– the third is the Newman modularity [NEW 04], which measures the quality of a partition, here _Q_ , compared to a graph, and not with _P_.

### 11.5.2.1. _The Rand index corrected by chance_

The comparison of partitions is often based on pairs of the _X_ elements. Two elements _x_ and _y_ can be joined or separated in _P_ and in _Q_. Both partitions are compatible on pair ( _x, y_ ) if they are simultaneously joined or separated. However, they are incompatible if they are joined in one and separated in the other. Let _r_ be the number of pairs simultaneously joined, _s_ the number of those which are simultaneously separated, and _u_ (respectively _v_ ) those which are joined (respectively separated) in _P_ and separated (respectively joined) in _Q_. Traditionally, _π_ ( _P_ ) denotes the number of pairs of the _X_ elements that are joined in _P_.

The Rand index [RAN 71], denoted _R_ , is the percentage of pairs for which there is compatibility:

[11.10]

it is therefore situated between 0 and 1. Clearly, 1− _R_ ( _P, Q_ ) is the symmetric difference distance between sets _π_ ( _P_ ) and _π_ ( _Q_ ).

In their article published in 1985, Hubert and Arabie observed that the Rand index does not have a zero expected value in random partitions. To correct this bias, they introduced the _Rand index corrected by chance_ , denoted here _R c_, which has zero expected value in this case.

The corrected Rand index is based on three values: the number r of connected pairs common to _P_ and _Q_ , their expected value _Esp_ ( _r_ ), under the hypothesis of random partitions having the same cluster cardinalities, as _P_ and _Q_ , and the maximum value _M ax_ ( _r_ ) of this index. They ended up with the formula:

[11.11]

with   and  . This maximum value is debatable, since the number of common joined pairs is necessarily limited by inf{| _π_ ( _P_ )|, | _π_ ( _Q_ )|}, but it ensures for _R c_ a maximum value equal to 1 only when the two partitions are identical. However this index can take on negative values which correspond to cases where P _c_ and _Q_ are further away than random partitions.

### 11.5.2.2. _Transfer distance_

The transfer distance between partitions is derived from an evidence, by moving a vertex from one cluster to another, possibly empty, one can go from any partition to any other, so that   can be covered. This property was already used by Régnier [RÉG 83] to calculate a "median partition" from a set of partitions.

This is, in fact, an _editing_ distance between partitions, in which the only possible operation is the transfer of a single element. Day, in his 1981 article, extended the set of possible operations to define around 10 metrics, studied from the computational complexity point of view. And he indicated, without any more detail, that the computation of the transfer distance is polynomial. Later, we calculated its maximum value [CHA 05], and used this relationship in   to compare distance indices between partitions [DEN 06].

Let _P_ and _Q_ be two partitions of _X_ with _n_ elements having respectively _p_ and _q_ classes:

We will assume that _p_ ≤ _q_. The minimum number of transfers to go from _P_ to _Q_ , denoted _θ_ ( _P, Q_ ), is obtained by establishing a matching between the clusters of _P_ and those of _Q_ , which retains a maximum number of elements in the matched clusters; it is those which do not need to be moved. To do this, we begin by adding _p_ − _q_ empty clusters to _P_ , so that _P_ is also considered as a partition with _q_ clusters.

Let   be the mapping of _P_ × _Q_ →  , which assigns to any pair of clusters the cardinal of their intersection. Let _n i,j_ = | _P i_ ∩ _Q j_| and _n i_ = | _P i_| and   = | _Q j_|, represent the cardinals of the clusters and the sums of the rows and columns of the contingency table. Let Δ be the mapping which, for every pair of clusters ( _P i_, _Q j_), assigns the cardinal of the symmetric difference, written Δ _i,j_ . We get:

Therefore we, consider the complete bipartite graph _K q,q_ , its vertices corresponding to the clusters of _P_ and _Q_ , where the edges are weighted either by   or by Δ.

PROPOSITION 11.1.– _The mapping, which minimizes the number of transfers between two partitions with q clusters P and Q, corresponds to a maximum weighted matching W_ 1 _in K q,q weighted by_  ; _in addition θ(P, Q) = n − W_ 1.

**Table 11.6.** I _ntersections and symmetric difference tables of_ _P_ _and_ _Q_ _clusters_

There is a equivalent formulation of this result, given in Day's article [DAY 81]. Instead of considering the conserved part between _P_ and _Q_ , we minimize the moved part. The graph is therefore weighted by Δ and a transfer corresponds to two operations, removing one element from a cluster, and assigning it elsewhere.

PROPOSITION 11.2.– _The mapping which minimizes the number of transfers corresponds to a minimum weighted matching W_ 2 _in K q,q , and weighted by_ Δ; _in addition_  .

The construction of the bipartite graph of clusters is in _O_ ( _n_ 2). The problem of maximum (or minimum) weighted matching in a complete bipartite graph can be solved by an assignment method, which is well-known in Operational Research [KUH 55, KUH 56]. The algorithm has a polynomial complexity in _O_ ( _q_ 3). We will not go into further detail, since the details can be found, for example, in Faure _et al_. [FAU 00].

EXAMPLE.– Consider _P_ = (1, 2, 3|4, 5, 6|7, 8) and _Q_ = (1, 3, 5, 6|2, 7|4|8).

Table 11.6 describes the intersections and symmetric differences of _P_ and _Q_ clusters. Two optimal matchings are shown in bold. Both give _θ_ ( _P, Q_ ) = 4. To the maximum weighted matching for table   corresponds the series of transfers: (1, 2, 3|4, 5, 6|7, 8) → (1, 3|4, 5, 6|2, 7, 8) → (1, 3, 5|4, 6|2, 7, 8) → (1, 3, 5, 6|4|2, 7, 8) → (1, 3, 5, 6|4|2, 7|8).

This distance is very well adapted to partitions which are very close to each other, as Denoeud _et al_. showed [DEN 06]. It refines the results of the corrected Rand index, when it is close to 1. However, for averagely similar partitions, the transfer distance assumes varying values between _n_ /4 and _n_ /2.

### 11.5.2.3. _Newman's modularity_

The concept of _modularity_ was introduced by Newman & Girwan [NEW 04] to measure the quality of a partition in relation to a graph. A partition is adapted to the graph, if most of the edges are inside the clusters and with a few edges outside. The proposed formula is:

[11.12]

in which _e k_ is the percentage of edges inside cluster _P k_:

and _a k_ is the percentage of edges, which have at least one extremity in _P k_, the inter-cluster edges contributing 1/2 to each of the clusters of the ends:

This quantity   measures the fraction of intra-cluster edges minus the expected value, if the _m_ edges were randomly selected. It does not compare the partitions _P_ and _Q_ but it measures the quality of partition _Q_ in relation to the graph. This criterion is often used as an optimization one [DUT 05], and to determine the number of classes (the one which maximizes modularity) [NEW 04, PON 06]. According to our simulations [GUÉ 08], it works rather well for sufficiently dense graphs, and better than the classic Calinski-Harabash rule [MIL 85] or other rules based on the variation of the optimized criteria.

## 11.5.3. _Results_

The simulation protocol consists of randomly selecting graphs and calculating the gap between the initial partition _P_ , which generates the graph, and the partition _Q_ returned by _PartOpt_. The average values of the above criteria are calculated over 100 trials. To compare the Czekanowski-Dice formula to the one of Pons-Latapy, we take sparse graphs with 200 vertices distributed into 5 clusters. Depending on the values of _p i_ and _p e_, the probabilities of the internal and external edges, more or less difficult problems are generated. For graphs with 200 elements distributed into 5 balanced clusters, a maximum of 19,900 edges, are disributed on average into, 3,900 internal edges and 16,000 external edges. The probabilities _p i_ and _p e_ lead us to obtain on average _N e_ and _N i_ edges of the two types.

Table 11.7 shows the average number of edges obtained, as a function of the probabilities chosen for the internal edges ( _p i_) and the external ones ( _p e_). These probabilities are external values. With a fixed value of _p e_, increasing pi makes the problem easier, and with a fixed value of _p i_, increasing pe makes it harder. For _p i_ = 0.30 and _p e_ = 0.05, the three distances would obtain very good results.

### 11.5.3.1. _Unweighted graphs_

We have inserted the classical shortest path distance [BER 58] between the Czekanowski-Dice and Pons-Latapy distances. The shortest path is integer-valued for unweighted graphs. So do not use it for weighted graphs.

**Table 11.7.** _Average number of edges obtained as a function of the probabilities chosen for the internal (p i) and external (pe edges: Ni internal edges, Ne external edges and Na = Ni \+ Ne_

**Table 11.8.** _Quality criteria of partitions reconstructed from the Czekanowski-Dice, path length and Pons-Latapy distances as a function of the probabilities of the internal (p i) and external edges (pe)_

The corrected Rand index, as well as the modularity criterion, have an upper limit equal to 1; the larger their values are, the better the computed partitions. The values of _θ_ are measured by using a mean transfer percentage in relation to 100 elements. For the Pons-Latapy distance, we have carried out trials, where walks varied from 3 to 20 in length, and the best result is edited. The optimal length of the walks is indicated between brackets along with the optimum modularity value (which also corresponds to the optimum values of the other criteria). It is far from being homogeneous, which means that for a given graph, several values must be tested, and also   can be used to select the appropriate random walk length. Here the number of steps is the same as for the 100 trials.

For the probability values generating graphs, we obtain the average values, which are indicated in Table 11.8. The first conclusion is that the Pons-Latapy distance is clearly less efficient than the Czekanowski-Dice distance for sparse graph partitioning. The latter is slightly better than the path length metric, even for _p i_ = 0.30 and _p e_ = 0.10 since, with high values for the corrected Rand index, the transfer distance is significant. The second conclusion is that the three criteria corroborate, since they rank the distances in the same order.

**Table 11.9.** _Quality criteria of partitions reconstructed for the distances for weighted graphs. The weights of the internal edges are uniformly randomly selected in the interval [0.4; 1.0], and the external edges in [0.2; 0.8]_

### 11.5.3.2. _Weighted graphs_

For weighted graphs, how can we fix the weights in the simulations? If for each edge, we randomly select a weight, independent of the clusters we aim to find, we can add an incoherent information, which does not allow us to evaluate the way in which the distances take into account the weightings. This is why we assign weights to the edges in the interval [0.4; 1.0] when these are intra-cluster edges, and lower weights in the interval [0.2; 0.8] when these are inter-cluster edges. The average weight values (0.7 and 0.5) are not very different, and it is possible that an internal edge receives a much lower weight than an external one. These weights are assigned simultaneously with the random selection of the edges.

For our extensions of the Czekanowski-Dice distance, there are two formulae "Probabilities" and "Intensities", which occupy the two first parts of the table, while the third corresponds to the Pons-Latapy formula. For this latter, the number of steps is given in Table 11.9 (and we have verified that it was still the optimum value).

It is clear that the weights assigned to the edges, in accordance with the initial classes, help in their discovery, to a great extent. Even harder problems ( _p i_ = 0.10, _p e_ = 0.01) can provide very acceptable results, for the probabilities formula, the error is less than 4% . Here again, though less clearly, this _D p_ formula gives better results, than the two others, and the Pons-Latapy distance performs better than the intensity formula in the last three series. The intensity formula, if it does not establish as good partitions as the others, will create partitions that will have a modularity slightly higher than the others. It means that the computed partitions are better adapted to the graphs than those which generated them.

# 11.6. Conclusions

We do not claim to conclude that the superiority of the _PartOpt_ method, applied to the Czekanowski-Dice distance and its extensions to the cases of weighted graphs, is better than the Pons-Latapy method. This method has not been tested here; we have just borrowed their distance formula.

Since their work does not stop here, but rather has only just started, the relationship of their metric with the spectral approach in clustering is one of the most informative, and allows us to avoid the delicate computation of eigenvectors. The generalization of their distance between vertices to a distance between clusters, will allows them to adopt an ascending hierarchical method similar to Ward [WAR 63]. A true methodological comparison should include this algorithm.

By carrying out these simulations, we merely wished to ensure that we could keep our extended Czekanowski-Dice distances to calculate functional clusters in proteinprotein interaction graphs. And this is what we have done, mostly for unweighted graphs.

# 11.7. Acknowledgments

This text could not have been written without numerous collaborations. I could not have arrived at these conclusions without the help of I. Charon, L. Denoeud, O. Hudry (ENST Paris), P. Kuntz (Polytech Nantes), and B. Leclerc (EHESS) in Mathematics and Computer Sciences and also with essential help of A. Baudot, E. Becker, C. Brun, and B. Jacq (CNRS Marseille) in Biology.

During the period of this work, we received support from CNRS (ACI IMPBio, the GDR Operational Research and the PEPS ST2I); I would like to thank their evaluation committees.

# 11.8. Bibliography

[ANG 08] ANGELELLI J., BAUDOT A., BRUN C., GUÉNOCHE A., "Two local dissimilarity measures for weighted graph with application to biological networks", _Advances in Data Analysis and Classification_ , vol. 2, p. 3-16, 2008.

[BAD 03] BADER G., HOGUE C., "An automated method for finding molecular complexes in large protein interaction networks", _BMC Bioinformatics_ , vol. 4 p. 2, 2003.

[BAR 00] BARABASI L., "The large-scale organization of metabolic networks", _Nature_ , vol. 407, p. 651-654, 2000.

[BAT 99] BATAGELJ V., MRVAR M., "Partitioning approach to visualization of large graphs", _Proceedings of the 7th International Symposium on Graph Drawing_ , vol. 1731 of Lecture Notes in Computer Science, Springer, p. 90-97, 1999.

[BAU 06] BAUDOT A., MARTIN., MOUREN P., CHEVENET F., GUÉNOCHE A., JACQ B., BRUN C., "PRODISTIN Website: a tool for the functional classification of proteins from interaction networks", _BioInformatics_ , vol. 22, no. 2, p. 248-250, 2006.

[BEC 09] BECKER E., GUÉNOCHE A., BRUN C., "Système de classes chevauchantes pour la recherche de protéines multifonctionnelles", in RIVALS E., RUSU I. (eds), _JOBIM_ '09, p. 49-54, 2009.

[BER 58] BERGE C., _Théorie des graphes et ses applications_ , Dunod, Paris, 1958.

[BRO 06] BROHÉe S., VAN HELDEN J., "Evaluation of clustering algorithms for protein-protein interaction networks", _BMC Bioinformatics_ , vol. 7, no. 488, 2006.

[BRU 03] BRUN C., CHEVENET F., MARTIN D., WOJCIK J., GUÉNOCHE A., JACQ B., "Functional classification of proteins for the prediction of cellular function from a protein-protein interaction network", _Genome Biol_., vol. 5, p. R6, 2003.

[BRU 04] BRUN C., HERRMANN C., GUÉNOCHE A., "Clustering proteins from interaction networks for the prediction of cellular functions", _BMC Bioinformatics_ , vol. 5, no. 95, 2004.

[CHA 05] CHARON I., DENOEUD L., GUÉNOCHE A., HUDRY O., "Maximum transfer distance between partitions", _Journal of Classification_ , vol. 23, no. 1, p. 103-121, 2005.

[DAY 81] DAY W., "The complexity of computing metric distances between partitions", _Mathematical Social Sciences_ , vol. 1, p. 269-287, 1981.

[DEN 06] DENOEUD L., GuÉNOCHE A., "Comparison of distance indices between partitions", in BATAGELJ V. et al. (eds), _Proceedings of IFCS 2006, Data Science and Classification_ , Springer, p. 21-28, 2006.

[DIC 45] DICE L., "Measures of the amount of ecologic association between species", _Ecology_ , vol. 26, p. 297-302, 1945.

[DON 00] VAN DONGEN S., Graph clustering by flow simulation, PhD thesis, University of Utrecht, 2000.

[DUT 05] DUTCH J., ARENAS A., "Community detection in complex networks using extremal optimization", _Physical Review E, American Physical Society_ , vol. 72, p. 4, 2005.

[ENR 02] ENRIGHT A., VAN DONGEN S., OUZOUNIS L.A., "An efficient algorithm for largescale detection of protein families", _Nucleic Acids Research_ , vol. 30, no. 7, p. 1575-1584, 2002.

[FAU 00] FAURE R., LEMAIRE B., PICOULEAU C., _Précis de Recherche Opérationnelle_ , Dunod, Paris, 2000.

[FIC 84] FICHET B., LE CALVÉ G., "Structure géométrique des principaux indices de dissimilarité sur signes de présence-absence", _Statistique et Analyse de Données_ , vol. 3, p. 11-44, 1984.

[GIR 02] GIRVAN M., NEWMAN M.E.J., "Community structure in social and biological networks", _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 99, p. 7821-7826, 2002.

[GLO 97] GLOVER F., LAGUNA M., _Tabu Search_ , Kluwer, Dordrecht, 1997.

[GUÉ 03] Guénoche A., "Partitions optimisées: évaluation et comparaison", _Mathématiques et Sciences humaines_ , vol. 161, p. 41-58, 2003.

[GUÉ 04] GUÉNOCHE A., "Clustering by vertex density in a graph", in BANKS D. et al. (eds), _Proceedings of IFCS '04, Classification, Clustering and Data Mining Applications_ , Springer, p. 15-24, 2004.

[GUÉ 05] GUÉNOCHE A., "Comparing recent methods in graph partitioning", _in ICGT '05, Electronic Notes in Discrete Mathematics_ , vol. 22, p. 83-89, 2005.

[GUÉ 08] GUÉNOCHE A., "Comparison of algorithms in graph partitioning", _RAIRO_ , vol. 42, p. 469-484, 2008.

[HAN 97] HANSEN P., JAUMARD B., "Cluster analysis and mathematical programming", _Mathematical Programming_ , vol. 79, p. 191-215, 1997.

[HUB 85] HUBERT L., ARABIE P., "Comparing partition", _Journal of Classification_ , vol. 2, p. 193-218, 1985.

[KUH 55] KUHN H., "The Hungarian method for the assignment problem", _Naval Research Logistics Quarterly_ , vol. 2, p. 83-97, 1955.

[KUH 56] KUHN H., "Variants on the hungarian method for the assignment problems", _Naval Research Logistics Quarterly_ , vol. 3, p. 253-258, 1956.

[KUN 92] KUNTZ P., Représentation euclidienne d'un graphe abstrait en vue de sa segmentation, PhD thesis, Ecole des Hautes Etudes en Sciences Sociales, 1992.

[MIL 85] MILLIGAN G., COOPER M., "An examination of procedures for determining the number of clusters in a data set", _Psychometrica_ , vol. 50, p. 159-179, 1985.

[MOO 01] MOODY J., "Identifying dense clusters in large networks", _Social Networks_ , vol. 23, p. 261-283, 2001.

[NEW 04] NEWMAN M., GIRVAN M., "Finding and evaluating community structure in networks", _Physical Review E_ , vol. 69, no. 2, 2004.

[PON 06] PONS P., LATAPY M., "Computing communities in large networks using random walks (long version)", Journal of Graph Algorithms and Applications, vol. 10, no. 2, p. 191-218, 2006.

[PON 07] PONS P., Détection de communautés dans les grands graphes de terrain, PhD thesis, University of Paris 7, 2007.

[RAN 71] RAND W., "Objective criteria for the evaluation of clustering methods", _Journal of the American Statistical Association_ , vol. 66, p. 846-850, 1971.

[RÉG 83] RÉGNIER S., "Quelques aspects mathématiques des problèmes de classification automatique", _I.C.C. Bulletin (1965) Reprint in Mathématiques et Sciences humaines_ , vol. 82, p. 13-29, 1983.

[ROU 03] ROUGEMONT J., HINGAMP P., "DNA microarray data and contextual analysis of correlation graphs", _BMC Bioinformatics_ , vol. 4 p. 15, 2003.

[WAR 63] WARD J.H., "Hierarchical grouping to optimize an objective function", _Journal of the American Statistical Association_ , vol. 58, no. 301, p. 236-244, 1963.

* * *

1 Chapter written by Alain GUÉNOCHE.

# Chapter 12

# Detection of Disjoint or Overlapping Communities in Networks 1

# 12.1. Introduction

Large networks allow us to describe complex systems that are arising from different domains. In order to study and understand the structure of the relationships between their entities, we model them by graphs, vertices corresponding to the entities of the domain, and edges to the relations between entities. Thus, in biology, a protein-protein interaction (PPI) network is a graph in which any vertex is a protein and any edge is a potential interaction between two proteins, corresponding to a physical contact at a given moment, in one or several tissues [FOR 05, STA 04]. In sociology, graph represents individuals connected by the social relationships [WHI 05] such as friendship [ZAC 77] or working relations [NEW 01]. In computer science, the web network consists of a set of pages connected by the hyperlinks [KLE 01].

The existence of _communities_ in a network corresponds to the presence of vertex clusters that are more strongly connected among them than to others. They create the connected clusters which have a higher density than the graph as a whole. The identification of this type of structure is interesting from different points of view. On the one hand, it happens in several real networks and, most of the time, corresponds to a concrete meaning in terms of the organization. In PPI networks, for example, a community generally correspond to a set of proteins sharing the same cellular function. In this way, the community detection methods are precious tools for the functional prediction [BRU 03, BRU 04]. In social networks, these communities can represent a groups of individuals sharing common interests or activities, such as the famous co-publications list which places all computer scientists more or less far from Erdös [MOO 03]. In the web, groups of pages or strongly connected sites often cover similar topics [KLE 01] and the detection of these communities enables the improvements in search tools [FLA 02].

The community detection methods are the topic of many studies since the seminal chapter of Girvan and Newman [GIR 02]. Most of them determine a vertex partition optimizing some quality criterion over the partition set, defined from the structure of the graph (see [BRA 05] for a survey of classical criteria). Among them, _modularity_ is very natural, since it is closely related to the communities definition; see [GAE 07] and [FOR 07] for a detailed and critical study of this concept. This criterion indicates to some extent a partition of the graph vertices which corresponds to its communities. Informally, the modularity of a partition quantifies which difference between the proportion of the internal edges in the clusters and the same quantity under the hypothesis that the graph is not structured into communities. Maximizing the modularity criterion generally leads to suitable partitions on the real graphs [NEW 04b, DUC 05, CLA 04].

As most of the quality criteria, a major drawback of this criterion is inherent to the fact that its optimization over the whole set of vertex partitions is a NP-hard problem [BRA 08]. Therefore, the optimum value cannot be proved in a reasonable computation time as soon as large sized graphs are analyzed. So, approximated optimization methods must be used, leading to reachable partitions computed in polynomial time and expected to be close to the optimum.

The most common heuristics based on hierarchical clustering principles, are as follows:

– ascending methods (agglomerative) [NEW 04a, DON 04, PON 06, CLA 04, WAK 07]: starting from the atomic partition (set of singletons), two clusters are merged at each iteration. The joined clusters are those which maximize the modularity;

– descending methods (divisive) [NEW 04b, NEW 06, WHI 05] starting from the whole graph. At each iteration, a cluster is split into two disjoint sub-clusters following the same principle.

In one case or the other, the algorithm produces a hierarchy of communities, and we retain a partition with the desired number of clusters or composed of clusters which maximize the modularity. This is a very general process which can be applied to many other criteria. Alternatively, other approaches such as simulated annealing [REI 06, GUI 04] or extremal optimization [DUC 05] are based on a more direct optimization of the criterion.

The above-mentioned methods have the advantage of being efficient and relevant for the large graphs. However, they produce disjoint clusters, which are not always justified or desired. That is why we try to solve the problem of constructing, still through modularity optimization, either strict partitions, or partitions into overlapping clusters, also called as _class systems_. The overlapping class systems appeared in the 1980s in theoretical studies on the distance families; for more information, see [BRU 07]. Except for _pyramids_ [DID 86], related to the existence of a total order on the vertex set, the application of these models to the real data has not been used as hierarchical or partitioning methods.

In this chapter, we develop a unified approach for the construction of the class systems which aims at maximizing the modularity. In the second section, we show how Newman's modularity, defined on strict partitions, can be extended to the class systems. By the way, we obtain an equivalent criterion having integer values which allows us to carry out the fusions or fissions of the clusters. In the third section, we develop a fusion-fission algorithm, which is used to find the strict partitions from the edges as the initial covering vertex set. In the fourth section, we extend this to overlapping clusters only applying the fusion procedure, but starting from several initial covering sets. The performances of the algorithms are compared to other modularity optimization methods, on the random graphs with known overlapping clusters or communities.

# 12.2. Modularity of partitions and coverings

Let us consider a simple undirected and unweighed connected graph, without loops, _G_ = ( _V, E_ ) and with vertices | _V_ | = _n_ , and | _E_ | = m edges. Let _P_ = { _V_ 1, _V_ 2, .. _V p_} be a partition of V into p clusters. We aim at detecting the communities in G, searching for a high-modularity partition, in polynomial time and space. The modularity of a partition is the difference between the proportion of the internal edges (within clusters) and what is expected under the hypothesis that this network is not structured into communities. More formally, let _e ij_ be the percentage of edges having one end in cluster _V i_ and the other end in cluster _V j_ , we have:

The probability that a randomly selected edge has an extremity in class _V i_ is therefore:

and the modularity of the partition P is defined by:

[12.1]

This formula has been modified in several equivalent ways in the literature (see for instance [DUC 05, NEW 06]). In the following, we refer to the formula given in [NEW 06]'s equation [18]:

[12.2]

where ( _A xy_) is the adjacency matrix of _G_ , _d x_ is the degree of vertex _x_ , and _α xy_ is the square matrix of order _n_ , which depends on _P_ such that:

[12.3]

Although M depends on _G_ , and _α_ on _P_ , we do not mention them in the notation.

The modularity is an increasing function of the edge density inside the clusters; it takes values between −1 and 1. It quantifies the quality of a partition of a graph so that it is frequently used for partitioning the algorithms on a given graph. Of course, it acts as a criterion to be maximized in order to find the best partition into communities of _G_ , in terms of modularity. Unfortunately, the maximization of equation [12.2] on the whole set of partitions of _V_ is a NP-hard problem, so that the optimum partition cannot be obtained for the large graphs ( _n_ > 500). To overcome this problem, several heuristics have been proposed. We first establish an equivalent criterion which leads to an integer value of the criterion to be optimized. It is based on a simple reformulation of equation [12.2]:

[12.4]

Obser _V i_ng that _A xx_ = 0 and _α xx_ = 1, we get:

[12.5]

where:

[12.6]

The quantities _K_ ( _G_ ) and _B xy_ rely on the intrinsic properties of _G_ and do not depend on _P_ . On the other hand, _α xy _depends only on _P_ . When the relation _α_ is transitive, it designates a partition _P_ :

Therefore, _Q_ ( _P_ ) is an affine function of modularity _M_ ( _P_ ) and depends only on _G_. Optimizing _M_ over the whole set of partitions of _V_ thus reduces to optimize thecriterion:

[12.7]

where _B xy_ and _α xy_ are, respectively, defined by equation [12.6] and [12.3].

This expression allows a better understanding of the behavior of the _Q_ modularity:

– _Q_ has integer values, as it is the sum of the weights of the merged pairs; this corresponds to a non-negligible saving in terms of memory;

– matrix α is defined for all the class systems, whether it is a partition or a covering;

– when the two clusters _V i_ and _V j_ are merged, the only modified values in _Q_ are those corresponding to the elements _x_ ∈ _V i_ and _y_ ∈ _V j_ . Thus, the sum of the corresponding values _B xy_ is added to _Q_. The modularity value increases if this sum is positive;

– _Q_ has an upper limit which is the sum of the positive values of _B_ :

Thus, _Q max_ is reached for all the class systems whose clusters are exclusively formed by the pairs ( _x, y_ ) with a non-negative value of _B_ , such as the maximal cliques of _G_.

In the next two sections, we propose the algorithms leading to approximate solutions for the optimization of [12.7].

# 12.3. Partitioning method

The algorithm introduced in [NEW 04a] in order to maximize the modularity of strict partitions is composed of two stages. In the first one, the starting point is the set of singletons. As there is no internal edge, the so-called _atomic_ partition has a null modularity. At each step of the algorithm, and as long as the modularity increases, the two clusters, leading to a resulting partition with a maximum increase of modularity, are merged. The algorithm stops when the clusters can no longer be merged without decreasing the modularity. In the second stage, a transfer procedure selecting the other assignments by giving the modularity gains, leads to a partition which is no longer in the hierarchy.

The first stage has the great advantage of automatically fixing the number of clusters of the computed partition. However, starting from the minimum value of modularity _Q_ , it depends on the choices made during the first iterations; as in any greedy algorithm, it never comes back to the choices previously done. This is the aim of the second stage.

In our algorithm, we explore an alternative way which starts from the maximum _Q max_, corresponding to a class system made of all the edges of _G_ having a positive weight in _B_. We first consider the covering of V defined by the edges: _R_ (0) = {( _x, y_ )| _B xy_ ≥ 0} providing an optimal solution of the covering optimization problem. Our algorithm converges towards a partition by constructing a series of coverings { _R_ (0), _R_ (1), . . . , _R_ ( _t_ ), . . .} until a partition _P_ of _V_ is obtained. To expect _Q_ ( _P_ ) to be as close as possible to _Q max_, at each step we attempt to iteratively merge or split the two overlapping clusters of _R_ ( _t_ ).

## 12.3.1. _Fusion and/or fission of clusters_

The proposed method combines both the ascending and the descending approaches. It is based on an iterative process of the _fusion-fission_ , meaning that during each step we either merge the two clusters or separate them, by assigning to one the elements of their intersection. We start from a particular covering of the graph, its initial clusters being defined by the set of edges. This covering maximizes the modularity function which will therefore decreases along with the iterations. During each step t of the algorithm, we choose a pair of overlapping clusters and one operation - fusion or fission - leading to the lowest decrease in modularity; therefore it provides a covering _R_ ( _t_ +1) with the maximum value among all the tested fusions or fissions.

More precisely, for every pair ( _V i_, _V j_ ) of clusters from _R_ ( _t_ ) such that _V i_ ∩ _V j_ ≠ ∅, we compute the fusion and fission gains of these two clusters where:

– fusion is the simple replacement of two clusters by their union;

– fission consists of replacing the two overlapping clusters by two others disjoint clusters   and   so that:

(i) V ′ _i_ ∪ V ′ _j_ = _V i_ ∪ _V j_ , V ′ _i_ ∩ V ′ _j_ = ∅

(ii) _V i_\ _V j_ ⊂ V ′i, _V j_ \ _V i_ ⊂ V ′ _j_

We then test the resulting modularity of these operations:

– in the case of fusion, it only concerns the pair of vertices ( _x, y_ ) such that ( _x, y_ ) ∈ _V i_ ∪ _V j_ are newly merged;

– in the case of fission, it concerns the inter-cluster edges of _V i_ × _V j_ .

Although the fusion of _V i_ and _V j_ is unique, there are many possible fissions, as many as bi-partitions of _V i_ ∩ _V j_ . Calculating the best fission into two clusters is a NP-hard problem [BOC 06]. Among the several simulation tests described in the following section, we have selected the two approximation procedures:

– ( _P_ 1): let us consider _x_ as an element of _V i_∩ _V j_ and _M i_( _x_ ) (respectively _M j_ ( _x_ )) its contribution to the modularity of the cluster _V i_ (respectively _V j_ ): _M i_( _x_ ) = ∑ _y_ ∈ _V i_ _B xy_ . If _M i_( _x_ ) > _M j_( _x_ ) (respectively _M j_ ( _x_ ) > _M i_( _x_ )), then _x_ is assigned to _V_ ′ _i_ (respectively   );

– ( _P_ 2): we use the bi-partitioning method proposed by [NEW 06]. The subdivision _V i_ ∪ _V j_ is produced by performing a spectral decomposition of the sub-graph induced by the cluster. This subdivision is performed according to the components (positive or negative) of the main eigenvector of the corresponding laplacian matrix.

_Comments_ : An improvement in the final partition can be obtained through an extra local optimization of the modularity criterion. In particular, we can apply a descent method, such as for example the Kernighan-Lin algorithm proposed in KER 70] (this algorithm is described in [Chapter 2), two partitions being defined as neighbors as soon as each of them can be obtained from the other by a transfer, thereby moving a single vertex from one cluster to another.

## 12.3.2. _Algorithm complexity_

Let (  _t_ ) _t_ ≥0 be the sequence representing the number of clusters at step _t_ of the algorithm. Since a fusion replaces the two clusters by a single cluster and a fission exchanges the two overlapping clusters by the other two disjoint,  _t_ ≤  0 ≤ _m_. At step 0, communities are edges, thus the number of pairs of overlapping communities is limited by _δ_ × _m_ , where _δ_ is the maximum degree of the graph.

At each step, we calculate the fusion and fission gains of one or two new cluster(s) (depending on the selected operation) with each of the existing communities (at most _m_ ). Each operation takes a  ( _n_ 2) time. Since there are  ( _δ_ _m_ ) steps, the whole algorithm runs in a time  ( _δ_ _n_ 2 _m_ 2). However, the PPI graphs are sparse, which means that m =  ( _n_ ). Consequently, the algorithm runs in  ( _δ_ 2 _n_ 4). In addition, a list to memorize the details of the overlapping pairs of communities is made. This list uses  ( _δm_ ) +  ( _nm_ ) +  ( _n_ 2) memory space. Since a community contains at the most n vertices and that  _t_ ≤ _m_ , the algorithm requires  ( _n_ × _m_ ) memory space. The computation of costs associated with a fusion and a fission requires  ( _n_ 2) operations, but only the result (and not the detail of each calculation) is memorized, therefore the algorithm uses a  ( _δm_ ) +  ( _nm_ ) +  ( _n_ 2) space. Since we have _δ_ ≤ _n_ ≤ _m_ , the space complexity of the algorithm is  ( _n_ × _m_ ), which corresponds to  ( _n_ 2) for the sparse graphs.

## 12.3.3. _Simulations_

We have compared the performances of our algorithms, which are named as FF1 and FF2 for methods (P1) and (P2), to those of two classical heuristics, the hierarchical algorithm (G) (also described in the introduction section) and the spectral algorithm (S) of modularity optimization respectively proposed by Newman in [NEW 04a] and [NEW 06].

### 12.3.3.1. _Tested graphs_

Some graphs habing a progressive community structure are generated by Newman'sprotocol [NEW 04a]. Each graph contains 128 vertices, distributed into 4 clusters of 32 elements; it will be denoted as the _reference partition_  . Let _z in_ (respectively _z out_) be the average inter-community (respectively extra-communities) degree, verifying _z_ = _z in_ \+ _z out_ = 16. Consequently, _z out_ can be kept as the only parameter of the generating process; it involves the value of _z in_ which sets the probabilities _p in_ and _p out_ of the intra and inter-cluster edges. The parameter _z out_ controls the separation degree of the communities: a low value leads to the graphs having many internal edges and few external ones, making the communities easily detectable in this case. The difficulty of the partitioning problem increases as _z out_ becomes higher.

### 12.3.3.2. _Criteria_

Let _P_ = ( _P_ 1, . . . , _P K_) be the partition computed by one of the above mentioned algorithms. In order to evaluate the quality of _P_ , we measure the similarity between _P_ and _P ref_ owing to the different measurements:

_The modularity ratio_ : Since the algorithms are supposed to optimize the modularity, it seems natural to use the _Q_ modularity as efficiency measurement. However, it is not proven that the reference partition achieves the global maximum of modularity; we therefore use the quotient:

[12.8]

_The percentage of well-clustered vertices_ : A more qualitative measurement of _P_ is the percentage of well-clustered vertices relatively to _P ref_ . We know that this criterionhas many drawbacks (see [DAN 06]). In particular, even though its significance is clear when _K_ = 4, it cannot be easily transposed to general cases. Here, we propose an alternative equation; we extend the partition with lowest number of communities by empty clusters and further establish a matching between the reference clusters and the calculated ones:

[12.9]

where _S_ is the set of all permutations _σ_ of {1, . . . , max( _K_ , 4)}.

_The corrected Rand index_ : The Rand index measures the similarity between the two partitions belonging to the same set. It is the number of pairs simultaneously joined or separated in both partitions. To obtain a null expected index value on random partitions, Hubert & Arabie, [HUB 85] established the _Rand index corrected by chance_. Setting _r_ the number of joined vertex pairs in the two partitions _P_ and _P ref_ and _π_ ( _P_ ) (respectively _π_ ( _P ref_)) the number of joined cluster pairs in _P_ (respectively _P ref_), the corrected _R_ and index is defined by:

[12.10]

where:

### 12.3.3.3. _Results_

For each value of _z out_ we randomly select a graph according to the probabilities, _p in_ and _p out_. We then run the algorithms FF1, FF2, G, S and calculate the above-mentioned criteria for the computed partition _P_. Figures 12.1 to 12.3 show, for each algorithm, the average values obtained over 100 graphs.

**Figure 12.1.** _Modularity ratio of the partitions computed by algorithms FF1, FF2, G, and S as a function of the average external degree_

**Figure 12.2.** _Percentage of well-clustered vertices by FF1, FF2, G, and S as a function of the average external degree_

**Figure 12.3.** _Corrected Rand index of the partitions obtained with algorithms FF1, FF2, G, and S_

First, we observe in Figure 12.1 that the average modularity ratio of the obtained partitions exceeds 1 as soon as _z out_ is larger than 8. This means that the modularity of these partitions (and therefore the optimal one) is greater than that of the reference partition. Here, we come up against a difficulty related to the simulation process; we expect to find out the optimum partition but we only compare the computed partitions to the reference one which is not optimal for many random graphs. In order to be relevant, the reference partition should be optimal. Figure 12.1 shows that this is not true as soon as _z out_ is high. In the remainder of this section, only the results obtained when _z out_ ≤ 8 will be considered as relevant. With the exception of S, the modularity ratio is nearly the same for all the algorithms on this interval; it is slightly better for FF1.

Figures 12.2 and 12.3 clearly show that the performance of the algorithms decreases when _z out_ increases from 0 to 8. This result was predictable since the inter-community connectivity increases with _z out_, leading to a more difficult optimization problem. Nevertheless FF1, followed by FF2 are more efficient than G and S, this last algorithm being the less effective of them all.

# 12.4. Overlapping partitioning methods

We have justified the resorting to the overlapping classes in the case of biology and protein-protein interaction networks; let us provide an example. The human alpha-crystallin protein is at the same time a structural component of crystallin and is implicated in the response to the thermal shock when it is expressed in other tissues. Such multifunctional protein allows an understanding of the complexity of certain phenotypes or the secondary effects of certain drugs. If we are looking for functional clusters, it is not relevant to assign each protein to a single class.

We have embarked on a research program for the detection of these multifunctional proteins [BEC 09]. It comes after several attempts to construct the overlapping clusters through the other approaches. The first one belongs to the _density-based clustering_ approach, for which a density function for each vertex is first evaluated. This may correspond to the rate of the edges or triangles in a more or less large neighborhood around each vertex. The general strategy involve in constructing the clusters around the local maxima of the density function. The assignment is done gradually, as long as there is no ambiguity. Clusters grows around these _cores_ and a vertex which is adjacent to a single current cluster is assigned to it. However, if a vertex is adjacent to several clusters, we have the choice to assign it to a single cluster, the one to which it is mostly connected, or to all of them, thereby creating the overlapping clusters.

This method was introduced in the thesis of T. Colombo [COL 04, COL 07] and extended the chapter written by L. Denœud _et al_. [DEN 05]. Several density functions have been tested and the results obtained on graphs whose edges correspond to the _orthologous_ relation between the genes (to be derived from a common ancestral gene without any duplication event), are quite satisfactory. However, they are not encouraging on sparse graphs. That is why we have decided to use the partitioning method developed earlier in order to establish the overlapping classes.

## 12.4.1. _Fusion of overlapping classes_

We have seen that the modularity formula _Q_ allows us to extend the ascending hierarchical process by starting with an overlapping class system. Actually, starting with any cluster set and applying only fusions lead to the clusters which will overlap, if and only if the initial set does. At first, two initial class systems have been studied:

– the maximum clique of the graph. In case where they are computable within a reasonable timeframe, they form a initial set of overlapping clusters with modularity _Q max_. Any fusion operation makes _Q_ decreasing;

– the edges of the graph. With the edges of the graph, we start from the same modularity value _Q max_ as cliques. Usually, the fusion process begins with some clique reconstruction, since fusion of the edges ( _x, y_ ) and ( _y, z_ ) maintains the modularity value as soon as ( _x, z)_ is also an edge. The modularity starts to decrease when there is no more pairs of clusters ( _V i_, _V j_ ) such that ∀ _x_ ∈ _V i_, ∀ _y_ ∈ _V j_ , ( _x, y_ ) ∈ _E_.

The efficiency of the hierarchical ascending algorithm depends on the number of initial clusters, since it determines the number of iterations. By starting with the cliques or edges, a large number of iterations are performed. In order to reduce the number of the initial clusters, the adjacency lists were also tested but without any satisfactory results. Then, a system of _individual cliques_ has been defined as follows:

– for each vertex _x_ a clique as large as possible is built. This is carried out by a polynomial time algorithm applied to each vertex. It establishes, in a number of steps bounded by _n_ , a maximal clique that contains _x_ without necessarily having the maximum cardinality in the set of cliques containing _x_. The process consists of adding vertices in the order of their relative degrees, as long as it makes a clique. After the elimination of the duplicates, the number of initial clusters is at most _n_.

During the fusion procedure, the modularity of the different class systems either decreases monotonically (for maximal cliques and edges) or increases then decreases (for individual cliques). In the fusion method described above, at each iteration, the selected clusters to merge maximize the modularity _Q_ ( _α_ ) of the resulting class system. The fusion of the two clusters _V i_ and _V j_ leads to their replacement by a new cluster _V i_ ∪ _V j_ . Two modifications have been made, which are as follows:

– the choice of the pair of merged clusters leads to a _chain effect_ , namely the merging of elements one after the other with larger and larger clusters. To overcome

* * *

**Algorithm 12.1.** Computation of centered cliques

* * *

1: **procedure** Compute Centered Cliques

2: /* C contains the set of centered cliques */

3: _C_ ← {} 4: /* main loop */

5: **for all** _x_ ∈ _V_ **do**

6: _C x_ ← { _x_ }

7: _L x_ ← list of the neighbors of _x_

8: **for all** _y_ ∈ _L x_ **do**

9: evaluate the degree of _y_ in the subgraph corresponding to _L x_

10: **end for**

11: **Sort** _L x_ in decreasing order of relative degrees

12:

13: /* build the clique centered in _x_ */

14: **for all** _y_ ∈ _L x_ **do**

15: **if** ∀ _z_ ∈ _C x_, ( _y, z_ ) ∈ _E_ **then**

16: _C x_ ← _y_

17: **end if**

18: **end for**

19: /* storing */

20: if _C x_ not in _C_ **then**

21: _C_ ← _C x_

22: **end if**

23: **end for**

24: **end procedure**

* * *

this classical effect, we have modified the selection rule: the two joined clusters are those maximizing the _average_ value of the modularity variation. This facilitates the fusion of small clusters and the chain effect is strongly reduced;

– in order to leave the single class without finishing, and to obtain manipulable overlapping clusters, the fusion process is terminated by using a parameter fixed by the user. Stopping criteria have therefore been introduced, either fixing the desired number of clusters, or setting up an upper bound for the cluster cardinality.

## 12.4.2. _Simulations_

In order to check that our method actually finds the overlapping class systems and assigns some vertices to several clusters, we have generated three types of random graphs in which the clusters are represented by squares in Figure 12.4:

– the first set (on the left) is made of graphs with 200 vertices distributed into 4 disjoint clusters of 50 elements, adding another overlapping cluster made of 10 elements taken from each of these clusters, thereby containing 40 _multiple_ elements;

**Figure 12.4.** _Diagram of three sets of graphs tested in simulations_

– the second set (in the middle) is made of graphs with 210 vertices distributed into 5 clusters of 50 elements; the first four classes are perfectly separated, while the fifth one is made of 10 elements taken from each of these classes, plus 10 vertices which belongs only to the latter;

– the third set (on the right) is made of graphs having 210 vertices. The first four clusters are disjoint and have 45 vertices; the fifth is made of 5 vertices taken from each of these four (20 from the intersections) and 30 additional extra vertices.

From these five clusters, we randomly select the edges with a probability pi and thus obtain the graphs showing more or less dense community structures. We apply on these graphs the fusion algorithm, starting successively from the edges, the maximal cliques and the centered cliques, until five clusters are obtained. Then, we compare them to the initial clusters by the way of four parameters:

– _NBC_ , the number of well-clustered elements, which belong to both computed cluster and corresponding reference cluster;

– _Mult_ , the number of multiple elements, clustered at least twice;

– _F P_ , the rate of false positives, that are elements clustered several times, while they only belong to a single cluster;

– _Rec_ , the number of recovered multiple elements, that are initially clustered several times.

For these graphs, the results are promising:

– for the first set (Table 12.1), the simulations show that the clusters are better recovered when starting from maximal cliques rather than the other class systems. They are correctly identified when _p i_ > 0.2 and the rate of false positives and multiple recovered vertices are acceptable;

– Simulations with the second set of graphs (Table 12.2) show that the initial clusters are efficiently recovered from the centered cliques and that the rate of multiple elements is correct. However, the rate of false positives gets significant, even if it remains less than 25% for the three initial class systems. The centered cliques are the most efficient of

**Table 12.1.** _Average values of the criteria over 100 random graphs generated according to the first set of graphs_

**Table 12.2.** _Average values of the criteria over 100 random graphs generated according to the second set of graphs_

**Table 12.3.** _Average values of the criteria over 100 random graphs generated according to the third set of graphs_

the three systems for _p i_ = 0.15, and determines a correct number of multiple elements despite some errors;

– the third set of simulations (Table 12.3) generates more difficult problems. The three class systems correctly find the initial clusters as soon as _p i_ ≥ 20%. The number of multiple elements is well predicted by the individual cliques. However, these elements are not all identified.

# 12.5. Conclusion

In this chapter, we have studied the modularity criterion as a tool to detect the high edge density clusters in graphs structured this way. We have introduced the modularity optimization algorithms for the detection of disjoint or overlapping communities in a graph. Rather than sticking to the value of this criterion on public graphs, we have established the simulation protocols in order to measure the efficiency of our algorithms and to compare the results in the average. For that task, we have selected several partitioning quality criteria. Comparisons made on the randomly generated graphs allow us to attest of their efficiency in terms of community detection. For strict partitions, the fusion/fission method has shown to be slightly better than the classic algorithms. For overlapping partitions, simulations indicate that it is possible to start from a system with a limited number of classes. In addition, we applied this method to a large human protein-protein interaction graph.

A high quality interaction network made of 27, 276 interactions for 9, 596 proteins has been extracted from the APID database (bioinfow.dep.usal.es/apid/). In this network, each protein interacts on average of 7.8 with others. The individual clique system allows to deal with it. In order to avoid the classes too large to share a common biological function, we limited the cluster cardinality to 200 proteins. The hierarchical method merges the 5,372 initial cliques into 77 final classes. Looking for the major functions in these clusters, by the way of _Gene Ontology_ , functional annotations allowed us to recover 65% of the known to be multifunctional proteins and to suggest the unknown functions of the other proteins described above.

# 12.6. Acknowledgments

We would like to thank Christine Brun (CNRS, Marseille) who inspired us to study the overlapping clustering problem. She is now leading a team devoted to protein interaction networks. This work was funded by the ACI IMPBio program and the PEPS 2008-2009 grants of the ST2I Department of the CNRS.

# 12.7. Bibliography

[BEC 09] BECKER E., GUÉNOCHE A., BRUN C., "Système de classes chevauchantes pour la recherche de protéines multifonctionnelles", Rivals E., Rusu I. (eds), _JOBIM_ '09, Nantes, p. 49-54, 2009.

[BOC 06] BOCCALETTI S., LATORA V., MORENO Y., CHAVEZ M., HWANGA D., "Complex networks: structure and dynamics", _Physics Reports_ , vol. 424, no. 4-5, p. 175-308, 2006.

[BRA 05] BRANDES U., ERLEBACH T., "On finding graph clusterings with maximum modularity", _Network Analysis: Methodological Foundations_ , vol. 3418 of Lecture Notes in Computer Science, Springer, 2005.

[BRA 08] BRANDES U., DELLING D., GAERTLER M., GÖRKE R., HÖFER M., NIKOLOSKI Z., WAGNER D., "On modularity clustering", _IEEE Transactions on Knowledge and Data Engineering_ , vol. 20, no. 2, p. 172-188, 2008.

[BRU 03] BRUN C., CHEVENET F., MARTIN D., WOJCIK J., GUÉNOCHE A., JACQ B., "Functional classification of proteins for the prediction of cellular function from a protein-protein interaction network", _Genome Biology_ , vol. 5, p. R6, 2003.

[BRU 04] BRUN C., HERRMANN C., GUÉNOCHE A., "Clustering proteins from interaction networks for the prediction of cellular functions", _BMC Bioinformatics_ , vol. 5, p. 95, 2004.

[BRU 07] BRUCKER F., BARTHÉLEMY J., _Eléments de classification: aspects combinatoires et algorithmiques_ , Hermès, Paris, 2007.

[CLA 04] CLAUSET A., NEWMAN M.E.J., MOORE C., "Finding community structure in very large networks", _Physical Review E_ , vol. 70, p. 066111, 2004.

[COL 04] COLOMBO T., Algorithmes pour la recherche de classes de gènes en relation fonctionnelles par l'analyse de proximités et de similarités de séquences, PhD thesis, University of Aix-Marseille II, 2004.

[COL 07] COLOMBO T., GUÉNOCHE A., "Looking for high density zones in a graph", BRITO, P. et al. (eds), _Selected Contributions in Data Analysis and Classification_ , Springer, p. 193-201, 2007.

[DAN 06] DANON L., DÍAZ-GUILERA A., ARENAS A., "The effect of size heterogeneity on community identification in complex networks", _Journal of Statistical Mechanics: Theory and Experiment_ , vol. 2006, no. 11, 2006.

[DEN 05] DENOEUD L., CHARON I., GUÉNOCHE A., HUDRY O., "Overlapping clustering in a graph and application to protein interactions", _ALIO/EURO Conference on Combinatorial Optimization_ , Paris, 2005.

[DID 86] DIDAY E., "Orders and overlapping clusters in pyramids", de Leeuw J. et al. (eds), _Multidimensional Data Analysis,_ DSWO Press, Leiden, p. 201-234, 1986.

[DON 04] DONETTI L., MUÑOZ M., "Detecting network communities: a new systematic and efficient algorithm", _Journal of Statistical Mechanics: Theory and Experiment_ , vol. 10, p. P10012, 2004.

[DUC 05] DUCH J., ARENAS A., "Community detection in complex networks using extremal optimization", _Physical Review E_ , vol. 72, p. 027104, 2005.

[FLA 02] FLAKE G.W., LAWRENCE S., GILES C.L., COETZEE F.M., "Self-organization and identication of web communities", Computer, vol. 35, no. 3, p. 66-71, 2002.

[FOR 05] FORMSTECHER E., ARRESTA S., COLLURA V., HAMBERGER A., MEIL A., TREHIN A., REVERDY C., BETIN V., MAIRE S., BRUN C., JACQ B., ARPIN M., BELLAICHE Y., BELLUSCI S., BENAROCH P., BORNENS M., CHANET R., CHAVRIER P., DELATTRE O., DOYE V., FEHON R., FAYE G., GALLI T., GIRAULT J., GOUD B., DE GUNZBURG J., JOHANNES L., JUNIER M., MIROUSE V., MUKHERJEE A., PAPADOPOULO D., PEREZ F., PLESSIS A., ROSBACH M., ROSSÉ C., SAULE S., STOPPA-LYONNET D., VINCENT A., WHITE M., LEGRAIN P., WOJCIK J., CAMONIS J., DAVIET L., "Protein interaction mapping: a drosophila case study", _Genome Research_ , vol. 15, p. 376-384, 2005.

[FOR 07] FORTUNATO S., BARTHELEMY M., "Resolution limit in community detection", _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 104, p. 36, 2007.

[GAE 07] GAERTLER M., GÖRKE R., WAGNER D., _Significance-Driven Graph Clustering_ , vol. 4508, Springer, Heidelberg, 2007.

[GIR 02] GIRVAN M., NEWMAN M.E.J., "Community structure in social and biological networks", _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 99, p. 7821-7826, 2002.

[GUI 04] GUIMERA R., SALES-PARDO M., AMARAL L.A.N., "Modularity from fluctuations in random graphs and complex networks", _Physical Review E_ , vol. 70, p. 025101, 2004.

[HUB 85] HUBERT L., ARABIE P., "Comparing partition", _Journal of Classification_ , vol. 2, p. 193-218, 1985.

[KER 70] KERNIGHAN B., LIN S., "An efficient heuristic procedure for partitioning graphs", _Bell System Technical Journal_ , vol. 49, no. 2, p. 291-307, 1970.

[KLE 01] KLEINBERG J., LAWRENCE S., "The structure of the web", _Science_ , vol. 294, p. 1849-1850, 2001.

[MOO 03] MOODY J., WHITE D.R., "Structural cohesion and embeddedness: a hierarchical concept of social groups", _American Sociological Review_ , vol. 68, no. 1, p. 103-107, 2003.

[NEW 01] NEWMAN M.E.J., "The structure of scientific collaboration networks", _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 98, p. 404-409, 2001.

[NEW 04a] NEWMAN M.E.J., "Fast algorithm for detecting community structure in networks", _Physical Review E_ , vol. 69, no. 6, p. 066133, 2004.

[NEW 04b] NEWMAN M.E.J., GIRVAN M., "Finding and evaluating community structure in networks", _Physical Review E_ , vol. 69, no. 2, p. 026113, 2004.

[NEW 06] NEWMAN M.E.J., "Modularity and community structure in networks", _Proceedings of the National Academy of Sciences USA_ , vol. 103, p. 8577-8582, 2006.

[PON 06] PONS P., LATAPY M., "Computing communities in large networks using random walks (long version)", _Journal of Graph Algorithms and Applications (JGAA)_ , vol. 10, no. 2, p. 191-218, 2006.

[REI 06] REICHARDT J., BORNHOLDT S., "Statistical mechanics of community detection", _Physical Review E_ , vol. 74, no. 1, p. 016110, 2006.

[STA 04] STANYON C.A., LIU G., MANGIOLA B.A., PATEL N., GIOT L., KUANG B., ZHANG H., ZHONG J., FINLEY JR R.L., "A Drosophila protein-interaction map centered on cell-cycle regulators", _Genome Biology_ , vol. 5, no. 12, p. R96, 2004.

[WAK 07] WAKITA K., TSURUMI T., "Finding community structure in mega-scale social networks", WWW '07: _Proceedings of the 16th international conference on Worldwideweb_ , ACM Press, New York, p. 1275-1276, 2007.

[WHI 05] WHITE S., SMYTH P., "A spectral clustering approach to finding communities in graphs", _SIAM Data-Mining Conference_ , Society for Industrial Mathematics, 2005.

[ZAC 77] ZACHARY W.W., "An information flow model for conflict and fission in small groups", _Journal of Anthropological Research_ , vol. 33, p. 452-477, 1977.

* * *

1 Chapter written by Jean-Baptiste Angelelli, Alain Guénoche and Laurence Reboul.

# Chapter 13

# Multilevel Local Optimization of Modularity 1

# 13.1. Introduction

Graphs and networks provide a natural representation of systems composed of interactive elements, and they find applications in a large number of fields, ranging from biology to social sciences along with information theory. If the empirical study of graphs has been historically limited to small-sized systems, researchers and engineers are now confronted with networks of millions, or even billions of vertices, due to the computer revolution. In such systems, a brute-force graphic representation of all the vertices and links does not provide insight about the system's organization, and different methods have thus been developed to produce useful and comprehensible information. Such methods aim at discovering regularities or hidden structures, which can then be used, both to create diagrams of the architecture and to identify often unknown functions or relationships between the vertices.

Several methods are based on the uncovering of the modular structure of large graphs, namely their organization into modules (also called communities), such that most of the links are concentrated within the modules, while there are fewer links between vertices belonging to different modules. It has been shown that these modules can help us to reveal common properties of their components, such as topics in information networks or functions in biological networks. In many of the cases, modules tend to be themselves modular. Such networks are called hierarchically

**Figure 13.1.** _A graph split into communities and the associated dendrogram_

modular, and are composed of modules at different scales. Figure 13.1 illustrates such a hierarchy, also called a dendrogram.

Several methods have been developed to discover modules in large graphs. In this chapter, we focus on so-called partitioning methods, where each vertex of the graph is assigned to one and only one community. At the core of most of these methods, there is a mathematical definition to evaluate the quality of such a partition. In most of the cases, such a quality function depends on the number of links inside of and/or between the communities. Several classic quantities have this form, such as cut and normalized cut, but also more recent quantities, such as Newman-Girvan modularity or different generalizations of modularity. For most of this chapter, we will concentrate on modularity, which has proven to be particularly efficient at revealing communities in heterogeneous graphs1 and we will briefly discuss other functions in section 13.5.

The main aim of this chapter is to introduce a greedy method to optimize modularity, which we called the Louvain method2. The main advantages of our method are: i) its speed, which allows us to analyze graphs of unprecedented size, which peaks upto several billion links; ii) its multiscale aspect, which leads to the detection of communities at different scales; iii) its excellent accuracy, when compared to other greedy methods. As we will show, our method also tends to circumvent inherent flaws of modularity due to its local aspect. The chapter is organized as follows. In section 13.2, we define the graph partitioning problem and the concept of modularity. In section 13.3, we focus on modularity optimization, present different types of methods proposed in the literature, and introduce our method. In section 13.4, we carry out a detailed comparison of our method with other methods. In section 13.5, we focus on advanced questions. In section 13.6, finally, we draw conclusions.

# 13.2. Basics of modularity

Consider an undirected graph _G_ = ( _V_ , _E_ ) with _N_ = | _V_ | vertices and _m_ = | _E_ | links. This graph can be described by its adjacency matrix _A_ , where the elements _A ij_ are the weights of the links between _i_ and _j_. In the case of an unweighted graph, the elements _A ij_ are equal to one when a link exists between _i_ and _j_ , and to 0 in the opposite case. Thereafter, we will concentrate on undirected graphs, and A is therefore a symmetric matrix. Finally, we observe that the values _A ii_ on the diagonal correspond to the weighted loops of the vertex _i_.

A community in a graph _G_ is the subgraph generated by a set of vertices _C_ of _G_. A partition of the graph _G_ into communities assigns each vertex _i_ to a single community _C i_ such that the two vertices _i_ and _j_ belong to the same community, if and only if _C i_ = _C j_. From now on, we will note   such a partition into communities. Most of the partitioning methods are based on a definition of the quality of a partition, namely a function _Q_ ( ) such that _Q_ ( 1) > _Q_ ( 2) if  _1_ is a better partition than  2. The so-called modularity introduced by Newman and Girvan [NEW 04b] is such a function. It is based on the intuitive idea that a community is a group of vertices which are strongly linked among themselves, but less to the nodes outside of the community. Formally, the modularity of a partition is defined as follows:

[13.1]

where the sum is carried out on all the pairs of vertices belonging to a same community, and where _k i_ = ∑ _j Aij_ is the degree of the vertex _i_. By construction, modularity compares, for each community, the number of internal links with the number of expected links for an equivalent random graph. The _null_ model is meant to represent, in an unbiased way, some of the known properties of the graph. The most popular choice of null model is a random graph, which has the same degrees as the original graph, i.e. the configuration model, where the expected weight of a link between _i_ and _j_ is given by   . By comparing the properties of the graph with those of a random graph, modularity does not simply favor dense modules, but rather modules denser than expected by randomness, when the observed node degrees are given. This definition can be equivalently rewritten as:

[13.2]

where   is an over all sum of the communities of the partition. Finally, the modularity of a partition is a real number ranging between −1 and 13.

Modularity has the advantage of being able to compare partitions with a different number of modules. Incidentally, this quantity was initially introduced to select the most adapted level of partitioning in the hierarchical decomposition of a network. Subsequently modularity emerged as an essential element of a large number of _clustering_ methods. These methods aim at optimizing the modularity of a graph, i.e. finding the partition that has the maximal value of _Q_. An exhaustive optimization of _Q_ is impossible, due to the explosion in the number of ways of partitioning a graph, when the size of a graph increases. It has been shown that the optimization of modularity is an NP-complete problem [BRA 06]. For this reason, several heuristics have been proposed to find high quality partitions and we will, in the following section, introduce the most popular types of methods. For a more detailed comparison of different methods in terms of quality and complexity, we can refer to [DAN 05].

Before describing some of these optimization methods, let us discuss some of the interesting properties of modularity. The two following properties are particularly simple to determine, but are nevertheless informative:

PROPERTY 13.1.– _A partition, where all the vertices are grouped into the same community, has a modularity equal to zero. This can be simply shown from the definition of the null model_  , _for which_ ∑ _ij_   = 2m, _and from the expression of modularity in this particular case:_

[13.3]

This property implies that any partition with a positive modularity is better than this trivial one, but also that it is always possible to find a partition such that Q ≥ 0.

PROPERTY 13.2.– _If a partition contains a disconnected community, it is always preferable (in terms of modularity) to split this community into connected communities. Let us consider, for the sake of simplicty, the case of a disconnected community C_ 1, _which is formed by two connected subgraphs C_ 11, _C_ 12. _In this case, modularity is given by:_

[13.4]

Given that _A ij_ = 0 _if i_ ∈ _C_ 11, _j_ ∈ _C_ 12, the sum ∑ _i_ ∈ _C_ 11,j∈ _C_ 12 is composed uniquely of negative terms, and it is thus preferable to split the community into two subcommunities.

This property implies that any partition made of disconnected communities is sub-optimal, and that the optimal partition of a graph is made of connected communities only.

# 13.3. Modularity optimization

## 13.3.1. _Existing methods_

In this section, we will briefly introduce different types of techniques used to maximize modularity. The efficiency of these methods is evaluated for sparse graphs, whose number of edges behaves like _O_ ( _Nlog_ ( _N_ )), and where _N_ is the number of vertices. Most of the algorithms attempt to find the best compromise between computation time and accuracy. Depending on the size of the network under scrutiny or the required precision, different methods can be chosen. Let us also note that several methods not only provide a partition with good modularity, but a whole hierarchy of communities.

Simulated Annealing (SA) is one of the slowest methods. This classic metaheuristic was introduced to optimize modularity by Guimera _et al_. [GUI 05]. SA consists of exploring the space of possible partitions by Monte-Carlo sampling. The transitions between partitions is done with a probability 1 in case of modularity gain, and _exp_ (− _β_ Δ _Q_ ) otherwise, where Δ _Q_ is the change in modularity. _β_ is a measure of stochastic noise and decreases as the exploration progresses. The presence of noise ensures that the method does not get trapped in a local maximum. Asymptotically, the method finds a partition that can be brought arbitrarily close to the optimal solution. Unfortunately, this precision incurs a cost, and it is not conceivable to use SA for networks with more than a few hundreds vertices.

Algorithms with intermediate speed allow us to optimize modularity in _O_ ( _N_ 2 _log_ ( _N_ )). Among these, one finds spectral methods, such as the method recently proposed by Newman [NEW 06], but also the Pons and Latapy method (PL) [PON 06]. PL is based on an exploration of the graph with random walkers. This exploration leads to a definition of distance dij between vertices _i_ and _j_ as that of the probability that a walker goes from _i_ to _j_ in a certain number _t_ of steps. The number of steps is a parameter, which must be chosen large enough to explore a significant part of the network, but not too large to avoid weakening of the modular structure. After evaluating these distances, vertices are grouped by applying hierarchical _clustering_ to the distance matrix, and modularity is used to select the best partition in the associated dendrogram.

Finally, greedy methods have been introduced to analyze networks whose size requires a linear algorithm. Such methods are agglomerative and successively cluster communities to form larger communities, when modularity increases after this fusion. The first method of this type was introduced by Newman [NEW 04a], which was then improved by Clauset _et al_. [CLA 04]. Until a few years back, it was the fastest method and it enabled the analysis of networks, where the networks were composed of millions of vertices. Unfortunately, this algorithm produces poor results in terms of modularity, primarily since it tends to form unequally sized modules. Moreover, this bias toward unbalanced partitions results in the tendency to significantly slow down the computational speed of the algorithm. To overcome this problem, Wakita and Tsunumi proposed a new version of the algorithm, which favors the formation of equally-sized modules [WAK 07].

## 13.3.2. _Known limitations_

Apart from the technical difficulties related to the optimization of modularity, most of the modularity-related methods suffer from three fundamental problems, which are actually related to the definition of modularity itself. An initial problem is encountered when the graph is hierarchically modular, and is composed of significant partitions at different scales. By design, the optimization of modularity will only find one of these partitions, thus neglecting the other significant levels. In other words, a singlescale approach such as modularity optimization is not satisfactory when the network is multiscale. Other approaches have been proposed to overcome this limitation. One approach consists of searching for local maxima of modularity [SAL 07], and not just the global maximum, which can be done through simulated annealing, for example. Another approach consists in introducing a resolution parameter into the definition of modularity [REI 04]. We will return to this point in section 13.5.

A second problem, related to the previous one, is the resolution limit of modularity [FOR 07], namely the fact that modularity tends to uncover partitions whose modules have a certain characteristic size depending on the size of the system. More precisely, modularity tends to favor the larger modules when the total number of links increases. This loss of resolution originates from the null model _k ikj_/2 _m_ , which has an explicit dependence on the total number of links. By construction, the contribution of a pair _i_ , _j_ to the modularity of a partition:

[13.5]

depends on the difference between two terms. The first term is simply the weight of the link between _i_ and _j_. The second term depends on the total size of the system, and decreases as the number of links increases (i.e. 2 _m_ ), with _A ij_, _k i_ and _k j_ fixed. The resulting resolution limit can have unpleasant consequences, for example by making undetectable significant communities which are smaller than a certain size, this limits the capacity of the modularity function to measure efficiently the quality of a partition. This problem is open for discussion, and suggests that the optimal partition in terms of modularity might not always be the most pertinent. In practice, however, this limitation is often overcame by introducing a resolution parameter into the definition of modularity, as given in section 13.5.

Finally, it has been shown that the optimal partition of a random graph, i.e. a graph in which no well-defined module is expected to be found, has a positive modularity and that it typically increases when the size of the system increases [GUI 04]. The modularity of a graph is thus not an absolute measure of the modular organization of a graph. In order to determine the importance of the modular structure of a graph, it has been proposed to compare the modularity _Q G_ of this graph with the modularity of an equivalent sized random graph. The z-score, can for example, evaluate the statistical importance of _Q G_:

[13.6]

where  _random_ is the mean modularity of an equivalent random graph and _σ random_ is the standard deviation of this magnitude. A graph will therefore be highly modular when _z_ >> 1, whereas small values of _z_ indicate that the modularity of a graph is not statistically significant.

## 13.3.3. _Louvain method_

In this section, we describe our approach for maximizing modularity in very large graphs [BLO 08]. We will show that our method out performs pre-existent or greedy methods in terms of speed and precision, and that its accuracy is also very close to that of significantly slower methods. Finally, we discuss the possibility to use it to uncover modules at multiple scales, when dealing with hierarchical systems, and as a way to overcome the aforementioned resolution limit issue.

Our approach consists of two phases, which are iteratively repeated, until a local maximum of modularity is obtained. The algorithm begins with an undirected weighted

**Figure 13.2.** _Each pass of the algorithm is split into two phases. The first phase consists in a local optimization (LO), where each vertex can join one of its direct neighbors community. The second phase consists in a merging of the vertices (VM), i.e to the construction of a metagraph whose vertices are the communities found at the end of the first phase. These passes are repeated until a modularity maximum is reached_

graph having _N_ vertices to which an index between 0 and _N_ − 1 has been randomly assigned. Algorithm 13.1 is a pseudo-code version of the method.

### 13.3.3.1. _First phase_

The initial partition consists of placing each vertex into a separate community, and this partition is therefore composed of _N_ singleton communities. We then consider the first vertex, i.e. with index 0, and calculate the modularity variation obtained by removing 0 from its community, and placing it in the community of one of its neighbors _j_. The modularity variation can be simply calculated by re-evaluating the contribution of the vertices from the communities of 0 and _j_ , as given in subsection 13.3.4. This variation, is therefore, calculated for each of the neighbors of 0, and the vertex 0 is then moved to the community where this increase is maximum, but only if this maximum increase is positive. If all the increases are negative, then the vertex 0 is placed back into its original community. This process is applied sequentially to all the vertices, and we call this an iteration. The process is then reapplied to all the vertices repeatedly, until no vertex is moved during a complete iteration. The first phase is then finished. We emphasize the fact that there are generally several iterations (i.e. after node _N_ − 1, one returns to node 0, and so on), and this phase ends when a local maximum of modularity is reached, which implies that no individual movement can increase the modularity. After this first phase, the network of _N_ vertices has been

* * *

**Algorithm 13.1**. Pseudo-code of the community detection algorithm

* * *

1: **procedure** COMMUNITY DETECTION( _G_ initial graph)

2: **repeat**

3: Place each vertex of _G_ into a single community

4: Save the modularity of this decomposition

5: **while** there are moved vertices **do**

6: **for all** vertex _n_ of _G_ **do**

7: _c_ ← neighboring community maximizing the modularity increase

8: **if** _c_ results in a strictly positive increase **then**

9: move _n_ from its community to _c_

10: **end if**

11: **end for**

12: **end while**

13: **if** the modularity reached is higher than the initial modularity, **then**

14: _end_ ← _false_

15: Display the partition found

16: Transform _G_ into the graph between communities

17: **else**

18: _end_ ← _true_

19: **end if**

20: **until** _end_

21: **end procedure**

* * *

divided in a partition   having _N c_ communities. If _N_ > _N c_, which implies that when the first phase has grouped some vertices, then the algorithm continues to the second phase, if not the algorithm is finished and the result is the partition _P_.

### 13.3.3.2. _Second phase_

The second phase involves construction of a new graph, whose vertices are the Nc communities discovered during the first phase. The weight of the link between two of these new vertices is given by the sum of the weights of the links, which existed between the vertices of these two communities [ARE 07]. The links which existed between the vertices of a same community create loops over the community in the new graph. Once this second phase is finished, it is possible to reapply the first phase of the algorithm on the weighted graph and iterate.

### 13.3.3.3. _Iterating both phases_

For the rest of this section, we will call a "pass" a combination of these two phases. The first phase consists of finding a local optimum, where each vertex can only be linked to one community in its direct neighborhood. The second phase consists of aggregating the vertices, such that the application of the first phase in the aggregate graph will lead to collective movements of vertices at a higher level. This repetition of passes naturally constructs a hierachy of communities which is reminiscent of the self-similarity nature of complex networks [SON 05] and naturally constructs a hierarchy of communities. The algorithm therefore outputs one partition per pass, and the average size of the communities and the modularity increase from one pass to another. We are interested in the partition which maximizes the modularity found during the last pass in section 13.4, as well as the hierarchy provided by the algorithm in section 13.5.

## 13.3.4. _Modularity increase_

The efficiency of the algorithm partly resides in the fact that the variation of modularity Δ _ij_ obtained by moving a vertex _i_ from its community to the community of one of its neighbors _j_ can be calculated by using the local information only. In practice, the variation of modularity is calculated by removing _i_ from its community Δ _remove_ ; _i_ (this is only done once), and then inserting it into the community of _j_ Δ _insert_ ; _ij_ for each neighbor _j_ of _i_. The variation is therefore: Δ _ij_ = Δ _remove_ ; _i_ \+ Δ _insert_ ; _ij_ .

### 13.3.4.1. _Remove a vertex from its community_

Let us calculate the variation of modularity when a vertex _x_ is removed from its community. We will assume that _x_ is not alone in its community (the opposite case is trivial). By removing x from its community, the size of the community of _x_ is decreased _C x_ → _C x_\\{ _x_ }, and a new community, which only contains _x_ is created _C′ x_

The original modularity is:

[13.7]

and after removing the vertex _x_ from _C x_, the modularity becomes:

[13.8]

where we used the fact that _A ij_ is symmetric. The modularity variation is given by:

[13.9]

### 13.3.4.2. _Inserting a vertex into a community_

Let us consider the situation where a vertex _x_ is alone in a community, and where it is moved into another community _C 1_. The original modularity is:

[13.10]

and after movement of _x_ to _C_ 1, which becomes  , and the modularity becomes:

[13.11]

The modularity variation is given by:

[13.12]

In both cases, whether it concerns removal or insertion, the calculations of variations are performed by using only local information on x and its neighbors.

## 13.3.5. _Convergence of the algorithm_

### 13.3.5.1. _Theoretical complexity_

The algorithm is heuristic and its computational speed is influenced by many parameters. The convergence of the algorithm is guaranteed in all cases, since the possible number of partitions is finite and there is a strict increase in modularity during each iteration. It is possible to determine the maximum number of stages in the case where the weights of the links are integers. In effect, the uppermost limit of the modularity is 1, and each movement leads to a increase in modularity, which is at least equal to:

[13.13]

Since, during each iteration, at least one vertex must be moved (no matter the pass), with the minimum increase indicated above, the maximum number of iterations is therefore in the order of 1/ _G min_, where each iteration is approximately linear in terms of the number of links. We deduce convergence in a maximum of _O_ ( _m_ 3) stages. As we will see below and in section 13.4, the algorithm is, in actual cases, linear or close, hence far from this theoretical complexity.

In section 13.4, we will present results on the accuracy of the algorithm, a problem for which numerous computer generated benchmarks have been proposed in the literature. In this section, we instead propose to use benchmarks to evaluate the speed of convergence of the algorithm. Such computer-generated graphs should comply with the following conditions: i) allow to rapidly tune the size of the system, i.e. the number of vertices; ii) have a strong modular structure and reproduce, as closely as possible, the properties of the real complex networks. To do so, we propose the use of a graph model where growth is driven by copying mechanisms, and which has been proposed to model social and biological networks [VAZ 03].

The graph is constructed as follows: the network is initially composed of one vertex and it grows by iteratively adding vertices. At each step, a new vertex is created and is connected to a pre-existing vertex, _i_ , which is chosen at random. With a probability _p_ , it also connects to all _k i_ neighbors of _i_. The new vertex is thus connected to either one or _k i_ \+ 1 vertices. This recursive model has the advantage of producing graphs with a high density of cliques, a power law distribution of degrees and the "small-world" property, i.e. a strong _clustering coefficient_ associated with a diameter that behaves like _log_ ( _N_ ). In addition, its construction is done in linear time with the number of added vertices.

Figure 13.3 shows the results obtained on random graphs constructed with this model for different values of _p_. The most important point is that the computation time is linear with the number of vertices for this class of graphs which, up to a constant, is optimal.

**Figure 13.3.** _Computation time as a function of the number of vertices for random graphs constructed with the copying model (average over_ 5 _experiments) for p_ = 0.2, 0.3 _and_ 0.4

# 13.4. Validation on empirical and artificial graphs

In this section, we will evaluate the performance of the algorithm and compare it with several existing approaches. These evaluations and comparisons will be carried out on artificial graphs, as well as on empirical complex networks, where the community structure of the artificial graphs is known. These two types of comparisons have their respective advantages and flaws. In the case of artificial graphs, the "natural" partition of the graph (or the partitions, for hierarchical graphs) is known, and a parameter generally allows us to control the coherence of the communities. A partitioning algorithm is validated by comparing the natural partition with the partition found by this algorithm.A popular quantity to compare two partitions  1 and  2 is normalized mutual information [DAN 05]:

[13.14]

where _N_ is the number of graph vertices, _N CD_ is the number of vertices belonging to the community _C_ of  1 and to the community _D_ of  2, and where _N C_ is the number of vertices in the community _C_ of a certain partition. The normalized mutual information is equal to 1, when the two partitions are identical, and equal to 0 when they are statistically independent.

Computing communities on artificial graphs has the main advantage that the natural partitioning of the graph is known. However, it has a drawback that it is based on a preconceived idea of what a community is4, and is also based on random graphs whose structure is very different from the one observed in nature. Partitioning algorithms are, therefore, in general, also applied to real-life complex network graphs. In this case, the natural partitioning, in general, is not known. It is, however, possible to evaluate the quality of a partitioning method through the modularity of the partitions found.

## 13.4.1. _Artificial graphs_

A standard way of validating community detection algorithms is to apply them to graphs, where the community structure of the graphs is already known. Let us introduce two such computer-generated _benchmarks_.

The first _benchmark_ [NEW 04b] generates graphs with one and only one level of communities: it is composed of 128 vertices, which are grouped into 4 clusters of 32 nodes. The average degree of each vertex is 16, and each vertex has on an average, _i_ links to vertices of other clusters and 16 − _i_ links to vertices of its own cluster. Smaller values of _i_ correspond to more modular graphs. An accurate algorithm should thus still be able to identify communities when i is high. The second _benchmark_ [ARE 06] has three levels, and consists of a graph with 512 vertices, where the vertices are grouped into 4 large clusters, which are themselves composed of 4 small clusters of 32 vertices. A vertex is linked with a vertex of its small cluster of 32 with a probability _p_ 0, with a vertex of its large cluster with a probability _p_ 1, and with another vertex with a probability _p_ 2. We assume that _p_ 0 > _p_ 1 > _p_ 2. To simplify, we assume here that each vertex has 16 links to its small cluster, 16 links to its large cluster and _i_ links to the rest of the graph. The parameter _i_ varies. The higher the value of _i_ , the fuzzier will be the 4 clusters. We will thus compare the uncovered partition with the natural partition into four clusters. In both cases, the partitions obtained by the algorithm and the predefined clusters are compared with the help of mutual information. A mutual information close to 1 is satisfactory.

Figure 13.4(a) illustrates the results obtained with our algorithm on the first benchmark. Our algorithm fully identifies the 4 clusters up to an external degree of 7. Beyond this value, mutual information rapidly decreases, and the output of the algorithm does not correspond anymore to the natural partition into four communities. Interestingly, it is precisely at this point that our algorithm starts to produce partitions with modularity that is higher than that of the partition into 4 clusters. As our method aims at maximizing modularity, it is therefore preferable to find those other

**Figure 13.4.** _For each benchmark, we have shown the modularity obtained by the algorithm, the modularity of the natural partition and the mutual information between the two partitions as a function of the benchmark parameter. Each point is the maximum value obtained over_ 100 _executions_

clusters. Figure 13.4(b) shows that similar results also occur in the second _benchmark_ , even though the difference in modularity between the optimal partition, i.e. the one optimizing modularity, and the natural partition, i.e. the one into 4 clusters, is very low when mutual information begins to decrease.

**Figure 13.5.** _Modularity of the natural partition, modularity given by the algorithm and mutual information between the two partitions, as a function of the benchmark parameter, for random graphs of size 1,000. Each point is an average over_ 24 _random graphs and the value obtained by the algorithm is the best over_ 100 _executions_

In the two previous _benchmarks_ , the graphs are characterized by a homogeneous distribution of degrees and communities have the same size. These two properties are clearly in contradiction with those of the real-life complex network graphs. Different authors have, therefore, proposed artificial graphs having more realistic properties [BRA 03, BAG 08]. We have tested our algorithm on the networks proposed by Lancichinetti et al. [LAN 08], which allow us to construct graphs that have a power law distribution of degrees and community sizes. These graphs are parameterized by the exponents _γ_ and _β_ of these distributions, and by the _mixing parameter μ_ which measures the fraction of inter-community links. For our comparison, we have chosen the same parameters, which were proposed in [LAN 08], that is to say _γ_ = 2 and _β_ = 1. The number of vertices of the graphs generated is 1, 000. Figure 13.5 shows the results on this _benchmark_ and, as before, mutual information remains high for a wide range of parameters. When mutual information starts to fall, we observe that the algorithm finds partitions having a significantly higher modularity than the one of natural partitions, which is the reflect of the aforementioned resolution limit. It is only for the value 0.7 of the parameter of the model that we observe a low mutual information, whereas the modularity difference is not really significant.

Finally, let us highlight the recent work of Lancichinetti and Fortunato [LAN 09], who carried out a detailed comparison of partitioning methods on the basis of this _benchmark_. Their work shows that our algorithm is the most efficient among modularity optimization methods, and is even much more efficient than simulated annealing, which is well-known for its excellent precision in terms of modularity. The relative drawback of simulated annealing has it origins in the resolution limit of the modularity, which does not allow detection of communities which are very heterogeneous in size. Their work therefore suggests that a greedy optimization, such as the one which we have adopted, helps at overcoming this problem. We will return to this point in section 13.5.

Even though they are more realistic than the two previous _benchmarks_ , the graphs of Lancichinetti and Fortunato are still very different from real-life complex network graphs, primarily due to the reason that each community is a random graph. Some of the key properties, such as the high density of triangles, and the presence of patterns at different scales or degree correlations, are therefore neglected. If it is possible to integrate such properties into random graphs, the number of parameters required renders this option impossible in practice.

## 13.4.2. _Empirical graphs_

In order to extend our tests to graphs that have a realistic structure, we have applied the algorithm to several empirical graphs, which are, in general, studied in the literature. These graphs cover a wide range of sizes and different types of systems (see Table 13.1):

– Zachary's _karate club_ graph [ZAC 77], a small social network with 34 vertices and 77 links, which has served as a test for several community detection algorithms;

– a graph between co-writers of articles coming from the Website _Arxiv.org_, having around 9, 000 vertices and 24, 000 links, obtained from [DAT 03];

– an Internet subgraph [HOE 03], with 70, 000 vertices and 351, 000 links;

– a mobile telephony graph corresponding to the aggregation of 6 months of telephone communications [LAM 08a], with 2 million users and 5.4 million links;

– and finally 3 web graphs of different sizes, one corresponding to the domain _nd.edu_ [ALB 99] with 325, 000 vertices and 1 million links, a subgraph of the domain . _uk_ with 39 million vertices and 783 million links and a graph with 118 million vertices and 1 billion links [BOL 04b, BOL 04a].

**Table 13.1**. Description of the graphs tested

Graphs | | _V_ | | | _E_ |
---|---|---
Karate
Arxiv
Internet
Web nd.edu
Telephony
Web uk-2005
WebBase 2001 | 34
9 × 103
70 × 103
325 × 103
2.04 × 106
39 × 106
118 × 106 | 77
24 × 103
351 × 103
1 × 106
5.4 × 106
783 × 106
1 × 109

In order to evaluate the precision and speed of our algorithm, we compared the modularity of its optimal partition and the computation time required with those obtained for other algorithms. Given the size of some of these networks, we did not consider slow algorithms such as simulated annealing, but only the fastest methods described in section 13.2. More precisely, our comparison is performed with the following algorithms:

– _walktrap_ , proposed by Pons and Latapy [PON 06], with a random walk length equal to _t_ = 5 and _t_ = 2;

– the CNM greedy modularity optimization algorithm proposed by Clauset _et al_. [CLA 04];

– the WT greedy algorithm proposed by Wakita and Tsurumi [WAK 07], which is a variation of CNM;

– a version of our algorithm, as shown in section 13.3, with a threshold _τ_ = 0.005 (see section 13.5), in order to accelerate the computations.

The results are shown in Table 13.2. They clearly show that our algorithm is much faster than those previously developed in the literature. This speed allows us to study systems having an unprecedented size. Even for these very large networks, the computation time remains reasonable (around 8 minutes for a network with a billion links). In fact, our only limitation in size is due to the storage of the graph in memory, and not to the computational complexity. However, the sequential nature of the algorithm, as it processes each vertex successively, allows us to overcome this limitation by reading the graph directly on the disk, and adjacency list by adjacency list. By doing so, the required memory space can be reduced to a great extent, so that we can process graphs of up to 15 billion links.

**Table 13.2**. _Results obtained on several graphs of increasing size with a stopping criterion τ = 0.005, seesection 13.5.3. Four approaches currently among the most efficient are tested. For each algorithm, the first column gives the modularity and the second the computation time. Our algorithm requires around_ 10 _Gb of memory to process the Web Base 2001 graph. This memory is mainly used for storing the graph itself. The two last columns correspond to the modularity of same-sized random graphs, the results being averaged over 20 executions_

Modularity is also higher than the one obtained by other methods. These differences can be significant, as in the case of the mobile telephony network, where the WT algorithm reaches its limits. The last line of the table shows the modularity obtained on simple random connected graphs having the same degree distribution as the corresponding empirical graphs [VIG 05]5. As the modularity of random graphs is much smaller than real-life complex network graphs, their modular structure if not very significant. It is also interesting to note that the computation of random graphs tends to be longer than that of real-life complex network graphs. This difference is due to the fact that the efficiency of our algorithm comes from the exploitation of structures at different scales in the graph. The absence of such structures in random graphs, therefore, significantly slows down the algorithm.

# 13.5. Discussion

In this section, we will cover three specific points of the algorithm. First, we will discuss the processing order of vertices and its influence on the results. Second, we will study the hierarchy produced by the algorithm, in order to validate the intermediate partitions. Finally, we propose several improvements to increase the efficiency of the algorithm, in terms of computation time and accuracy. We will also cover applications and possible generalizations of the algorithm.

## 13.5.1. _Influence of the processing order of vertices_

The algorithm described above is deterministic, except for two points: the order in which the vertices are processed during each iteration, and the choice which must be made if two communities offer the same modularity increase, during the movement of a vertex. Since the second case is not very frequent and can be solved by choosing, for example, a random movement, let us focus in depth on the first case. Our numerical experiments show that the modularity of the optimal partition only weakly depends on the node ordering. In most of the cases, the difference between the best and the worst modularity obtained over several experiments only amounts to a few percents (see Figure 13.10).

However, similar values of modularity do not imply that the optimal partitions are similar to each other, as there might exist a large number of local maxima whose modularity is very close to the true maximum of modularity, but still correspond to very different partitions. It has been shown that such situations occur when the modular structure of the graph is not well-defined, for example in random graphs, whereas the local maxima tend to be very similar when the modular organization is strong [KAR 08]. This idea was recently used to test the _robustness_ of the modular structure of a graph. In [RON 09], the authors use a variant of our algorithm to produce a large number of optimal partitions, by varying the order in which the vertices are processed. The robustness of the optimal partition is defined by the normalized mutual information between all the pairs of partitions.

Finally, we have considered simple rules to choose a _good ordering_ of the nodes, in order to improve the accuracy or the computation time of the algorithm, without conclusive results. In particular, we calculated the communities on a graph, then recalculated the communities on the same graph, by processing the vertices in the order of their assignment to the communities (all the vertices of community 1, then all those of community 2, etc.). Despite this supposedly more favorable choice, neither the computation time nor the optimum modularity were significantly different from a random order.

## 13.5.2. _Intermediate communities_

Resolution limit implies that the optimization of modularity is blind to communities whose size is smaller than  , where _m_ is the number of links. Modules of partitions optimizing modularity might therefore be made of smaller significant modules. A standard way to look for these subcommunities is to recursively resplit each community, until the subcommunities can no longer be divided. One advantage of our agglomerative algorithm is that it provides a set of intermediate partitions, one for each pass. In this section, we explore the relevance of these intermediate solutions, and show that they provide useful information about the system, when it is hierarchical.

### 13.5.2.1. _Resolution limit_

A typical example of the resolution limit problem is illustrated by the graph in Figure 13.6(a). The example is taken from [FOR 07]: the graph considered is a set of same-sized cliques connected in a ring by a single link. The optimization of modularity tends to merge communities, if their size is lower than  . In the current case, if the number of cliques is high, then merging the cliques 2 by 2 (or 3 by 3, or more) improves the modularity.

For this type of graph, our algorithm gives good results and, by selecting the right level, allows us to circumvent the resolution limit problem. To prove this point, let us consider a ring of 5-cliques. During the first pass, irrespective of the number of cliques in the ring, the algorithm always uniquely identifies the cliques (each community is one and only one clique). However, at higher levels, the algorithm merges the cliques to optimize modularity, as expected, due to resolution limit. Figure 13.6(b) shows the average size of the communities, as a function of the number of cliques. Up to 22 cliques, resolution limit is absent and the communities all have exactly 5 vertices.

**Figure 13.6.** _A typical example of a graph where the resolution limit phenomenon is in play_

Beyond this value of 22, merging of the cliques commences, first 2 by 2, then 3 by 3, etc6. Despite the fact that the algorithm produces non-intuitive communities at higher levels, the relevant cliques are available in the hierarchy provided by the algorithms.

**Figure 13.7.** _Correlation between the modularity of each community found by the algorithm, obtained by rerunning the algorithm, or obtained by observing the subcommunities provided by the algorithm, on the mobile telephony graph. Only levels 2 to 6 are indicated to improve readability_

### 13.5.2.2. _Resplitting of intermediate communities_

Let us explore in more detail the hierarchy of communities, which is produced by the algorithm, and verify that intermediate communities provide useful information. To do so, we have considered each community _c_ discovered by the algorithm during some pass _p_. We have then compared the modularity of the subgraph _c_ , when removed from the rest of the graph, with the modularity of the partition of _c_ obtained from pass _p_ − 1.

Results are shown in Figure 13.7 each community _c_ is associated with a point that shows the modularity of its optimal partition, and of the partition produced at the previous level. One expects the points to be on the diagonal, if intermediate partitions are perfect. Except for the first level, one observes that most of the communities are close to the diagonal, which suggests that intermediate levels uncover relevant sub-communities. At the highest levels, quality decreases, which is expected due to the accumulation of constraints at lower levels. The poor agreement of the first level is due to the fact that communities at the previous level are trivial, i.e. single nodes.

### 13.5.2.3. _Sales-Pardo model_

To further explore the validity of intermediate partitions, we also tested the algorithm on the hierarchical benchmark of Sales-Pardo [SAL 07], which is very similar to the one introduced in section 13.4. This benchmark is a random graph which is made of modules at different levels, which nest into each other. The network is composed of

**Figure 13.8.** _Mutual information of the two levels for the Sales-Pardo benchmark, as a function of the parameter ρ_. _The limit of_ 0.9 _is indicated_

640 vertices, which are made of 4 modules of 160 vertices, where each of them is made up of 4 modules of 40 vertices. Based on construction, the probability that a link exists between two vertices is higher, if they belong to the same smaller module. Module cohesion depends on a parameter _ρ_ , which determines the density of links across different modules at all levels. The test of our algorithm was performed as follows: for each value of _ρ_ , we produced 100 benchmarks and optimized modularity. We then measured the mutual information between: i) the first level found by the algorithm and the partition into 16 modules of 40 vertices; ii) the second level found by the algorithm and the partition into four modules of 160 vertices. These quantities were averaged over 100 realizations. Our simulations confirm that the intermediate levels are correctly identified, with a mutual information above 0.9 for the lowest level, over a broad range of values of _ρ_ , see Figure 13.8.

Three experiments – ring of cliques, resplitting of intermediate communities and hierarchical benchmark – suggest that intermediate partitions provide a way to circumvent resolution limit, and explore the hierarchical organization of the system. Our algorithm is thus an interesting alternative to computationally more expensive methods, in order to detect multiscale communities.

## 13.5.3. _Possible improvements_

Several simple solutions exist in order to improve the algorithm efficiency. In the next section, we will introduce in detail the primary solutions, and then briefly describe other minor modifications. These modifications cannot provide improvements

**Figure 13.9.** _Increase in modularity and proportion of moved vertices at each iteration during the first pass, on the mobile telephony graph_

in certain cases, but at additional costs. These different modifications have been tested but, for the sake of simplicity, only the first one was included in the online version of the code.

### 13.5.3.1. _Partial optimization_

A straightforward way to accelerate the algorithm is to stop the process, as soon as the increase in modularity, is below a certain threshold. This can be performed at two levels: when moving nodes within the passes, or when iterating the phases. Figure 13.9 shows the total modularity increase, as well as, the proportion of moved vertices for each iteration on the mobile telephony graph during the first pass. Both quantities are strongly correlated and rapidly decrease. The same behavior was observed on all the graphs considered: the decrease is exponential during the first iterations, and then becomes slower. Based on this observation, an heuristic consists of stopping the iterations, as soon as the increase is lower than a given value _τ_.

This threshold must be chosen carefully by balancing two trends: when chosen too high, the accuracy of the algorithm is expected to decrease; when chosen too low, the optimization will be slowed down by unnecessary iterations. Figure 13.10 shows modularity and computation time as a function of _τ_. As expected, increasing the value of _τ_ decreases the number of iterations, and thus decreases the computation time. Interestingly, the loss of quality is minimal, as long as _τ_ remains lower than 0.005 before rapidly increasing when _τ_ is higher than 0.01. This phenomenon, observed on several real and artificial graphs led us to use a stopping criterion _τ_ = 0.005 in all experiments.

**Figure 13.10.** _For the mobile telephony graph. Each value is averaged over_ 100 _executions_

### 13.5.3.2. _Removing leaves_

If the graph is unweighted and without loops, it can be show that it is always preferable to place a vertex of degree 1 in the same community, as the one of its only neighbor. It is, therefore, possible to remove such vertices before executing the first pass, and to reinsert them afterward. This operation must be done carefully, since the calculation of modularity takes into account the number of links. To do so, each vertex receives a loop, whose weight is the number of the removed adjacent nodes. This trick may lead to a non-negligible gain in time, especially for very large graphs with heterogeneous degrees, but it should be used with caution, as the cost of this additional operation might overweigh its gain. This trick has not been used for the experiments described here.

### 13.5.3.3. _Vertex moving_

Figure 13.9 shows that fewer and fewer vertices are moved during each iteration for a given pass. Since consideration of the movement of a vertex involves a cost, a simple heuristic no longer processes a vertex, which has been moved during a previous iteration. This is, of course, too restrictive, since as soon as a vertex is left in the same place, it can no longer move. Intermediate solutions are preferable, for instance freezing a vertex during a single iteration, and processing it with a certain probability. The vertex is only produced, if its neighborhood has moved, etc. These intermediate solutions must be studied in more depth.

### 13.5.3.4. _Returning to lower levels_

The successive passes of the algorithm are no longer based on the vertices that are considered individually, but rather on clusters of vertices. At this level, single vertices can no longer leave their community. The accuracy of the algorithm can be improved by authorizing vertex movements after a partition has been found, if they increase the overall modularity. This can be done after each pass or once at the end. The gain, in any case, is relatively small.

### 13.5.3.5. _Simulated annealing_

Several other modifications are possible, but one of the most natural is based on the concepts related to simulated annealing, by allowing for non-optimal movements, i.e. not always selecting the neighbor leading to a maximum increase in modularity, but any neighbor, with a probability depending on this modularity increase. This approach has not been tested (for more information on simulated annealing algorithms, see Chapter 6).

## 13.5.4. _Known uses_

### 13.5.4.1. _Analysis of large graphs_

The aim of this section is to highlight some of the applications of the algorithm. In the experiments and case studies described above, the authors applied the partitioning methods described in this chapter (or variations of it) to large graphs to discover intermediate descriptive levels, i.e. between the individual vertices and the global system, and verify their validity by using known information about the vertices:

– in [RAF 09], the authors study networks of citations between journals. The communities, obtained by the optimization of modularity, are compared with the subject of ISE and used to draw up maps of the science;

– in a similar work [WAL 09], the authors study networks of co-citations and show that the communities are in agreement with the known structure of scientific disciplines;

– in [LEW 09], a protein interaction network is studied and it is shown that the communities are homogeneous from a biological point of view, which opens up new perspectives to study proteins for which there is no known annotation;

– in [FEN 09], the authors propose the use of partitioning methods to study the correlations between exchange rates and structural changes in markets;

– in [MEU 09], finally, the authors used our multiscale method to discover the hierarchical structure of the human brain, based on fMRI data in a state of rest.

### 13.5.4.2. _Application to other cost functions_

The algorithm described in this chapter has been presented in the context of modularity optimization, but it can also be used to optimize other cost functions. Examples include the multiscale modularity proposed by Reichardt and Bornholdt [REI 04]:

[13.15]

where the parameter _γ_ gives more or less importance to the null model, and allows for a control of community resolution. The optimal partition is known to be composed of fewer and fewer communities (which will therefore be larger), when _γ_ is decreased7. A variation of our algorithm has also been used to optimize the "map equation" of a partition [ROS 08]. This quality function attempts to measure the length of code required to describe the trajectory of a random walk on the graph. Another generalization has been developed to optimize the stability of a partition [LAM 08b]. The stability is defined, for a directed or undirected graph, in terms of persistence of a random walk inside the communities. Finally, a variant of the algorithm has been proposed to partition the links of a graph [EVA 09], and thus discover the overlapping communities for the vertices.

# 13.6. Conclusion

In this chapter, we have presented a community detection method called the Louvain method, which aims at partitioning a graph into non-overlapping communities by optimizing Newman-Girvan modularity. However, other quality functions can also be considered. This method offers excellent results, in terms of computation time and accuracy, which has allowed us to efficiently process real-life complex network graphs of unprecedented sizes. The main limitation for the experiments that we performed was the storage of the network in the primary memory, rather than the computation time. However, this limitation can be circumvented by taking advantage of the sequential nature of the algorithm, and reading the graph sequentially on the hard drive. By doing so, we were able to partition a graph with more than 15 billion links and several hundred million vertices.

In addition to its excellent results and speed of execution, the method naturally provides a hierarchy of communities produced at each pass of the algorithm. We have shown that intermediate levels tend to provide a good representation of the sub-communities present in the system, and that they allow, up to a certain point, to compensate for the effect of the resolution limit. Finally, we introduced several simple modifications, which allow a decrease in computation time or an improvement in the quality of the algorithm. We studied in detail one of these modifications, which consists of prematurely interrupting the computation, when the increase in modularity is marginal, and showed that it allows a considerable acceleration of the algorithm, without altering the efficiency.

Certain aspects remain to be detailed, notably the role of the processing order of the vertices. While the variation in modularity obtained for different orders is very low, the partitions themselves can be very different. The possibility of using the variation between optimal partitions to characterize the robustness of the modular architecture is an interesting research direction. Generalizations should also be considered, for instance to uncover modules in directed or bipartite graphs.

# 13.7. Acknowledgments

This work was funded by an ARC grant of the French community of Belgium, by the Belgian network DYSCO financed by the _Interuniversity Attraction Poles_ program introduced by the Belgian state, by the European project MAPAP SIP-2006-PP-221003 and by the MAPE project (ANR France). Renaud Lambiotte would also like to acknowledge the support of the EPSRC.

The principle of the method outlined in this article was developed during the final project assignment of M.E. Lefebvre and is set out in [BLO 08].

# 13.8. Bibliography

[ALB 99] ALBERT R., JEONG H., BARABASI A.-L., "The diameter of the worldwide web", _Nature_ , vol. 401, p. 130-131, 1999.

[ARE 06] ARENAS A., DÍAZ-GUILERA A., Pérez-Vicente C., "Synchronization reveals topological scales in complex networks", _Physical Review Letters_ , vol. 96, no. 114102, 2006.

[ARE 07] ARENAS A., DUCH J., FERNANDEZ A., GOMEZ S., "Size reduction of complex networks preserving modularity", _New Journal of Physics_ , vol. 9, no. 176, 2007.

[BAG 08] BAGROW J., "Evaluating local community methods in networks", _Journal of Statistical Mechanics: Theory and Experiment_ , vol. P05001, 2008.

[BLO 08] BLONDEL V., GUILLAUME J.-L., LAMBIOTTE R., LEFEBVRE E., "Fast unfolding of communities in large networks", _Journal of Statistical Mechanics: Theory and Experiment_ , vol. P10008, 2008.

[BOL 04a] BOLDI P., CODENOTTI B., SANTINI M., VIGNA S., "Ubicrawler: a scalable fully distributed web crawler", _Software: Practice & Experience_, vol. 34, p. 711-726, 2004.

[BOL 04b] BOLDI P., VIGNA S., "The webgraph framework I: compression techniques", _Proceedings of the 13th International Worldwide Web Conference (WWW 2004)_ , Manhattan, USA, ACM Press, p. 595-601, May 2004.

[BRA 03] BRANDES U., GAERTLER M., WAGNER D., "Experiments on graph clustering algorithms", _LNCS_ , vol. 2832, p. 568-579, 2003.

[BRA 06] BRANDES U., DELLING D., GAERTLER M., GOERKE R., HOEFER M., NIKOLOSKI Z., WAGNER D., Maximizing Modularity is hard, ArXiv Physics e-prints physics/0608255, August 2006.

[CLA 04] CLAUSET A., MOORE C., NEWMAN M., "Finding community structure in very large networks", _Physical Review E_ , vol. 70, no. 066111, 2004.

[DAN 05] DANON L., DIAZ-GUILERA A., DUCH J., Arenas A., "Comparing community structure identification", _Journal of Statistical Mechanics: Theory and Experiment_ , vol. P09008, 2005.

[DAT 03] Cup Datasets C.K., 2003, http://www.cs.cornell.edu/projects/kddcup/datasets.html.

[EVA 09] EVANS T., LAMBIOTTE R., "Line graphs, link partitions, and overlapping communities", _Physical Review E_ , vol. 80, no. 016105, 2009.

[FEN 09] FENN D., PORTER M., MUCHA P., MCDONALD M., WILLIAMS S., JOHNSON N., JONES N., Dynamical Clustering of Exchange Rates, arXiv 0905.4912, 2009.

[FOR 07] FORTUNATO S., BARTHÉLEMY M., "Resolution limit in community detection", _Proceedings of the National Academy of Sciences_ , vol. 104, p. 36-41, 2007.

[GUI 04] GUIMERÀ R., SALES-PARDO M., AMARAL L., "Modularity from fluctuations in random graphs and complex networks", _Physical Review E_ , vol. 70, no. 025101, 2004.

[GUI 05] GUIMERA R., AMARAL L., "Cartography of complex networks: modules and universal roles", _Journal of Statistical Mechanics: Theory and Experiment_ , vol. P02001, 2005.

[HOE 03] HOERDT M., MAGONI D., "Completeness of the Internet core topology collected by a fast mapping software", in _11th IEEE International Conference on Software, Telecommunications and Computer Networks (softcom)_ , Split (Croatia), October 2003.

[KAR 08] KARRER B., LEVINA E., NEWMAN M., "Robustness of community structure in networks", _Physical Review E_ , vol. 77, no. 046119, 2008.

[LAM 08a] LAMBIOTTE R., BLONDEL V., de KERCHOVE C., HUENS E., PRIEUR C., SMOREDA Z., DOOREN P. V., "Geographical dispersal of mobile communication networks", _Physica A_ , vol. 387, p. 5317-5325, 2008.

[LAM 08b] LAMBIOTTE R., DELVENNE J.-C., BARAHONA M., Laplacian dynamics and multiscale modular structure in networks, arXiv 0812.1770, 2008.

[LAN 08] LANCICHINETTI A., FORTUNATO S., RADICCHI F., "Benchmark graphs for testing community detection algorithms", _Physical Review E_ , vol. 78, no. 046110, 2008.

[LAN 09] LANCICHINETTI A., FORTUNATO S., "Community detection algorithms: a comparative analysis", arXiv 0908.1062, August 2009.

[LEW 09] LEWIS A., JONES N., PORTER M., DEANE C., The function of communities in protein interaction networks, arXiv 0904.0989, April 2009.

[MEU 09] MEUNIER D., LAMBIOTTE R., FORNITO A., ERSCHE K., BULLMORE E., "Hierarchical modularity in human brain functional networks", _Front Neuroinformatics_ , 2009.

[NEW 04a] NEWMAN M., "Fast algorithm for detecting community structure in networks", _Physical Review E_ , vol. 69, no. 066133, 2004.

[NEW 04b] NEWMAN M., GIRVAN M., "Finding and evaluating community structure in networks", _Physical Review E_ , vol. 69, no. 026113, 2004.

[NEW 06] NEWMAN M., "Finding community structure in networks using the eigenvectors of matrices", _Physical Review E_ , vol. 74, no. 036104, 2006.

[PON 06] PONS P., LATAPY M., "Computing communities in large networks using random walks", _Journal of Graph Algorithms and Applications_ , vol. 10, p. 191-218, 2006.

[RAF 09] RAFOLS I., LEYDESDORFF L., "Content-based and algorithmic classifications of journals: perspectives on the dynamics of scientific communication and indexer effects", _Journal of the American Society for Information Science and Technology_ , vol. 60, p. 1823- 1835, 2009.

[REI 04] REICHARDT J., BORNHOLDT S., "Detecting fuzzy community structures in complex networks with a Potts model", _Physical Review Letters_ , vol. 93, no. 218701, 2004.

[RON 09] RONHOVDE P., NUSSINOV Z., "Multiresolution community detection for megascale networks by information-based replica correlations", _Physical Review E_ , vol. 80, no. 016109, 2009.

[ROS 08] ROSVALL M., BERGSTROM C., "Maps of random walks on complex networks reveal community structure", _Proceedings of the National Academy of Sciences of United Sates of America_ , vol. 105, p. 1118-1123, 2008.

[SAL 07] SALES-PARDO M., GUIMERA R., MOREIRA A., AMARAL L., "Extracting the hierarchical organization of complex systems", _Proceedings of the National Academy of Sciences_ , vol. 104, p. 15224-15229, 2007.

[SON 05] SONG C., HAVLIN S., MAKSE H.A., "Self-similarity of complex networks", _Nature_ , vol. 433, p. 392-395, 2005.

[VAZ 03] VAZQUEZ A., "Growing network with local rules: preferential attachment, clustering hierarchy, and degree correlations", _Physical Review E_ , vol. 67, no. 056104, 2003.

[VIG 05] VIGER F., LATAPY M., "Efficient and simple generation of random simple connected graphs with prescribed degree sequence", in COCOON, _The 11th International Computing and Combinatorics Conference_ , Kunming, China, p. 440-449, August 2005.

[WAK 07] WAKITA K., TSURUMI T., "Finding community structure in mega-scale social networks:

[extended abstract]", in _WWW '07: Proceedings of the 16th International Conference on Worldwide Web_ , Banff, Canada, ACM, p. 1275-1276, May 2007.

[WAL 09] WALLACE M., GINGRAS Y., DUHON R., "A new approach for detecting scientific specialties from raw cocitation networks", _Journal of the American Society for Information Science and Technology_ , vol. 60, p. 240-246, 2009.

[ZAC 77] ZACHARY W., "An information flow model for conflict and fission in small groups", _Journal of Anthropological Research_ , vol. 33, p. 452-473, 1977.

* * *

1 Chapter written by Thomas AYNAUD, Vincent D. BLONDEL, Jean-Loup GUILLAUME and Renaud LAMBIOTTE.

1 A graph is called heterogeneous if the distribution of degrees is not concentrated around its mean. For such systems, the mean degree is not representative, and fluctuations of degrees between the vertices play an important role. It has been shown that a large number of empirical graphs are heterogeneous [ALB 99].

2 The method was developed whilst all the authors were working at the University of Louvain in Belgium, the name remained even after several of the authors left the university. The C++ and Matlab code is available online at the address <http://findcommunities.googlepages.com/>.

3 In practice, the modularity cannot be equal to 1. An upper limit equal to 1 − 1/  is given in [FOR 07].

4 In the case of directed graphs, for example, the quality of a partition can be defined by the density of links inside the communities, or by the persistence of probability flux in these communities. Artificial graphs can be based on one or the other of these principles, and lead to incompatible tests [ROS 08].

5 For the two largest graphs, this method is not useable, we therefore used a method that does not guarantee connectivity, or the absence of multiple links. The latter are quite rare and therefore do not falsify the results.

6 Contrary to what the figure may suggest, communities have _on average_ 9 vertices for a number of cliques between 22 and 44, but their size is always a multiple of 5, since they are made of aggregations of 5-cliques.

7 Varying the value of _γ_ involves changing the number of links in the network in _m′ = m_ / _γ_.

# Appendix

# The Main Tools and Test Benches for Graph Partitioning 1

This appendix gives an idea about the reference tools for graph partitioning optimization which are developed by different laboratories throughout the world and even available in the Internet. However, this list is far from exhaustive. Besides the fact that they were designed to graph partitioning optimization, the common characteristic of these tools is that they all include a multilevel algorithm (the multilevel method is explained in Chapter 2).

The graph partitioning tools described in this appendix are sequential tools (not parallel). We can refer to Chapter 4 on graph partitioning parallelization for more information on parallel partitioning tools, and, in particular, the PT-SCOTCH [CHE 08] and PARMETIS [KAR 99] tools.

In this appendix, the presentation of graph partitioning optimization tools is classified into two main sections. Section A.1 lists the tools which solve the problem for constrained graph partitioning and section A.2 represents those for unconstrained graph partitioning1. The description of these tools is followed by a presentation of the test benches for graph partitioning which is explained in section A.3.

# A.1. Tools for constrained graph partitioning optimization

Engineering computations like numerical analysis and, in particular, matrix computations constantly require more power and speed because of the increase in the sizes of the matrix. In order to meet these requirements, parallel machines or clusters of computers are used. We must, therefore, distribute the computation load among these machines. A significant number of graph partitioning tools have been created to compute the partition required for these load distributions. The underlying partitioning problem is a constrained graph partitioning problem. Most of the tools discussed in this section are based on this perspective.

In this section, we present five libraries to find out the solutions for the constrained graph partitioning problem: CHACO, METIS, SCOTCH, JOSTLE and PARTY.

## A.1.1. _Chaco_

The CHACO library HEN 95a] was developed by Hendrickson and Leland from the _Sandia National Laboratories_ of the United States of America. It is available in the following web address: [www.cs.sandia.gov/∼bahendr/chaco.html.

It is the oldest of the libraries presented in this section. However, it has a wide variety of resolution techniques for constrained graph partitioning optimization:

– a spectral algorithm2 using the two resolution methods [POT 90, HEN 95b], the Lanczos algorithm or an algorithm associated with the iterative method of the Rayleigh quotient, and the multilevel method;

– a generalized Kernighan-Lin type refinement algorithm for _k_ -direct partitioning of a weighted graph and using the Fiduccia-Mattheyses implementation method. This algorithm is described in [HEN 95c];

– the multilevel algorithm described in [HEN 95c];

– an algorithm using an inertial method based on the geometric information in addition to the description of the graph [SIM 91, WIL 91].

The source code of this library is available. The CHACO library is no longer maintained and the current version of CHACO is version 2.0 which was published in October 1995. This version is entirely coded in C language, and it has an easy-to-use interface. All the methods implemented in CHACO are used to solve the constrained graph partitioning optimization problem. This application is highly configurable but suffers from a major drawback: the number of parts of the sought partition must be a power of two. One of the most important parameters is called KL_IMBALANCE and allows the modification of the partitioning balance of the sought partition.

## A.1.2. _Metis_

The METIS graph partitioning library KAR 98b] was developed by Karypis and Kumar from the University of Minnesota in the United States of America. The library, its documentation, and lot of other resources concerning METIS are available in the following web address: [glaros.dtc.umn.edu/gkhome/views/metis/.

METIS is one of the most commonly used libraries for graph partitioning optimization. The most recent stable version is version 4.0.1 which was published on the 30th of November 1998 and its latest applicant release is version 5.0rc1 which will be released on the 14th of June 2011. This library provides two softwares: _pMetis_ and _kMetis_. The first software, _pMetis_ , uses the recursive multilevel bisection algorithm as described in [KAR 98a]. The second software, _kMetis_ , is implemented with the direct k-partitioning algorithm shown in [KAR 98c]. The code of this library is fully available, although it is not free. It is coded in C language and has a very accessible interface. Unfortunately, this software has much less configuration than the CHACO software. For example, the partitioning balance is not configured. Thus, _pMetis_ searches for a 1.01 partitioning balance and _kMetis_ for a 1.03 partitioning balance, even if both these softwares provide partitions with biggest partitioning balances than those for some graphs.

## A.1.3. _Scotch_

The SCOTCH library PEL 08] is developed by Pellegrini from the INP Bordeaux in France. Everything about the Scotch library is available in the following web address: [www.labri.fr/perso/pelegrin/scotch/.

SCOTCH is composed of an abundant set of programs concerning the graph partitioning for numerical analysis. The current version of SCOTCH is 5.1.10 and was released on 2nd of September 2010. The SCOTCH provides the static mapping tools for partitioning or matrix ordering. Its cost function can be, in addition to the cut objective function, taken into account with the particular characteristics of the partitioning problem [PEL 96, HEN 97]. The SCOTCH partitioning tool uses a multilevel recursive bisection algorithm with a Kernighan-Lin type refinement algorithm as described in [PEL 96]. The library, entirely coded in C language, has various programming interfaces. Contrary to the other libraries, which use a CHACO graph format, SCOTCH uses its own graph format. However, a SCOTCH program allows an easy passage from one format to another.

One of the great advantages of the SCOTCH partitioning tool is its polymorphism. Indeed, the SCOTCH partitioning algorithm can be set precisely. In addition, it is not restrictive in terms of the partitioning balance to be taken into account.

## A.1.4. _Jostle_

The JOSTLE software WAL 02] was developed by Walshaw of the Greenwich University in the United Kingdom. The sources of the JOSTLE library are not available; only it is executable. In order to obtain a copy of the JOSTLE software, a signed copy of the license must be sent to Walshaw. Information related to this executable and its license are available in the following web address: [staffweb.cms.gre.ac.uk/ c.walshaw/jostle/.

The current version of JOSTLE is version 5.1 which was released on July 2005. The JOSTLE is a software based on the multilevel algorithm described in [WAL 00]. The scope and use of the JOSTLE software is reduced, from the fact that it does not have a programming interface. However, it is fairly parameterizable: the cardinal of the sought partition is not reduced to the powers of two as with Chaco; its partitioning balance is very adjustable, ranging from 1.00 to 1.50; and its refinement algorithm can be used independently.

## A.1.5. _Party_

The PARTY PRE 98] library is offered to Preis from the University of Paderborn in Germany. The Internet page of the PARTY library is available in the following web address: [www2.cs.uni-paderborn.de/cs/robsy/pary.html.

The PARTY library is fairly a typical partitioning software. Indeed, this application provides naturally eight partitioning methods, out of which one is optimal and two are region growing. In addition, the PARTY can use all the algorithms provided by the CHACO. The PARTY has two refinement methods:

– a Kernighan-Lin type algorithm using the Fiduccia-Mattheyses implementation;

– an algorithm using the _Helpful-Sets_ methods described in [DIE 95, SCH 03, MON 04].

PARTY also allows adjustment of the partitioning balance.

The PARTY library is no longer developed. Its previous stable version is version 1.1 which was published on 16th September 1996 and its latest test version is version 1.99.

# A.2. Tools for unconstrained graph partitioning optimization

There are few tools developed for unconstrained graph partitioning than for constrained graph partitioning. In addition, these tools are very often image segmentation tools which use the concept of distance between the vertices, and this makes them unusable to directly solve the graph partitioning optimization problem.

The most well-known image segmentation tool based on graph partitioning is developed by Jianbo Shi and Jitendra Malik [SHI 00]. Their software uses a spectral method3 which minimizes the normalized cut4.

## A.2.1. _Graclus_

The GRACLUS software is developed by Dhillon, Guan, and Kulis from the University of Texas in the United States of America DHI 07, DHI 04b, DHI 04a]. This software is a version of the _pMetis_ software in the case of unconstrained graph partitioning. The software and the articles describing the methods used by theGRACLUS software are available in the following web address: [www.cs.utexas.edu/users/dml/Software/graclus.html.

The current version of the GRACLUS software, version no. 1.2, was released on 10th of February 2009. The GRACLUS software is based, for partitioning, on the multilevel algorithm of _pMetis_ , and for refinement, on a _k_ -means algorithm. The _k_ -means algorithm is commonly used to cluster the individual points in an  _p_ pace by using an Euclidean distance. Thus, in order to use this algorithm, Dhillon, Guan, and Kulis propose in [DHI 07] a distance between the vertices of a graph.

# A.3. Graph partitioning test benches

Several test benches concerning the graph partitioning exist. The most popular and the widespread is no doubt but the graph partitioning archive of Walshaw from the University of Greenwich in the United Kingdom. The description of this archive is followed by those of other test benches.

## A.3.1. _Graph partitioning archives of Walshaw_

The Internet Website of Walshaw, entitled _the graph partition archive_ , contains a significant database of the graphs and partitions for the constrained graph partitioning optimization problem. This Internet Website is available in the following web address: staffweb.cms.gre.ac.uk/∼wc06/partition/.

This Website currently forms the largest test bench dedicated to the constraint graph partitioning in the world. Partitions found by different methods are compared in this Website.

This test bench contains 34 graphs of different nature (regular and simple/ordinary) coming from multiple sources. These graphs are shown in Table A.1. For each graph, the

**Table A.1.** _Detailed characteristics of the 34 graphs of Walshaw graph partitioning test bench_

Website provides the partitions of 2, 4, 8, 16, 32, or 64 parts obtained for the following partitioning balances: 1.00, 1.01, 1.03, and 1.05. These partitions correspond to the best partitions found by different constrained graph partitioning algorithms. In August 2009, 24 tools or constrained graph partitioning algorithms were compared with each other. Among them, several tools have already been introduced at the beginning of this appendix: CHACO, METIS, and JOSTLE. Thus, there are no less than 34 × 6 × 4 = 816 partitions available to consult and download.

This archive is mentioned in several articles of the literature. The assets of information that are found here allows a fairly good in deep evaluation of the efficiency of the constrained graph partitioning algorithms.

## A.3.2. _Other test benches_

Other test benches for graph partitioning are available in the different Internet web sites of graph partitioning tools as described above (see sections A.1 and A.2).

In Chapters 6 and , different algorithms are tested on regular graphs. Actually, they are not entirely regular, because a small proportion of their vertices does not have the same number of neighbors as the other graph vertices. These graphs, as well as others, are available in the following web address: perso.ec-lyon.fr/charles-edmond.bichot/.

Table A.2 shows the characteristics of the different graphs. Since these graphs have been constructed deterministically, their minimum cuts are known and shown in the table.

In addition to the significance of the minimum cut of each of these graphs, they are provided with a coordinate file. These coordinates are useful for visualizing 2D or 3D graphs. Of course, a partition of these graphs can also be visualized. To do so, the graphs

**Table A.2.** _Detailed characteristics of the regular graphs used inChapters 6 and _

**Figure A.1.** _Optimal bisection of the graphs Sphere-400 and Torus-200 × 500_

and partitions must be converted into the _Open Inventor_ format. This conversion is, among other things, possible owing to the graph partitioning library the Scotch. Once the _Open Inventor_ file is generated, it can be displayed in the free software _ivview_ 5. Figure A.1 illustrates the result of this visualization for the bisection of the two graphs: Sphere-400 and Torus-200 × 500.

# A.4. Bibliography

[CHE 08] CHEVALIER C., PELLEGRINI F., "PT-Scotch: a tool for efficient parallel graph ordering", _Parallel Computing_ , vol. 34, no. 6-8, p. 318-331, 2008.

[DHI 04a] DHILLON I.S., GUAN Y., KULIS B., "Kernel k-means, spectral clustering, and normalized cuts", in _Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , ACM, p. 551-556, 2004.

[DHI 04b] DHILLON I.S., GUAN Y., KULIS B., A Unified View of Kernel k-means, Spectral Clustering and Graph Partitioning, Report no. TR-04-25, University of Texas at Austin, 2004.

[DHI 07] DHILLON I.S., GUAN Y., KULIS B., "Weighted graph cuts without eigenvectors: a multilevel approach", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 29, p. 1944-1957, 2007.

[DIE 95] DIEKMANN R., MONIEN B., PREIS R., "Using helpful sets to improve graph bisections", in _Proceedings of the DIMACS Workshop on Interconnection Networks and Mapping and Scheduling Parallel Computations_ , DIMACS, p. 57-73, 1995.

[HEN 95a] HENDRICKSON B., LELAND R., The Chaco User's Guide, Sandia National Laboratories, 2.0 edition, 1995.

[HEN 95b] HENDRICKSON B., LELAND R., "An improved spectral graph partitioning algorithm for mapping parallel computations", _SIAM Journal on Scientific Computing_ , vol. 16, no. 2, p. 452-469, 1995.

[HEN 95c] HENDRICKSON B., LELAND R.W., "A multilevel algorithm for partitioning graphs", in _Proceedings of Supercomputing_ , ACM, 1995.

[HEN 97] HENDRICKSON B., LELAND R., DRIESSCHE R.V., "Skewed graph partitioning", in _Proceedings of the 8th SIAM Conference on Parallel Processing for Scientific Computing_ , SIAM, 1997.

[KAR 98a] KARYPIS G., KUMAR V., "A fast and high quality multilevel scheme for partitioning irregular graphs", _SIAM Journal of Scientific Computing_ , vol. 20, no. 1, p. 359-392, 1998.

[KAR 98b] KARYPIS G., KUMAR V., Metis: _A Software Package for Partitioning Unstructured Graphs, Partitioning Meshes, and Computing Fill-Reducing Orderings of Sparse Matrices_ , University of Minnesota, 4.0 edition, September 1998.

[KAR 98c] KARYPIS G., KUMAR V., "Multilevel k-way partitioning scheme for irregular graphs", _Journal of Parallel and Distributed Computing_ , vol. 48, no. 1, p. 96-129, 1998.

[KAR 99] KARYPIS G., KUMAR V., "Parallel multilevel k-way partitioning scheme for irregular graphs", _SIAM Review_ , vol. 41, no. 2, p. 278-300, 1999.

[MON 04] MONIEN B., SCHAMBERGER S., "Graph partitioning with the party library: helpfulsets in practice", in _Proceedings of the Symposium on Computer Architecture and High Performance Computing_ , IEEE, p. 198-205, 2004.

[PEL 96] PELLEGRINI F., ROMAN J., Experimental Analysis of the Dual Recursive Bipartitioning Algorithm for Static Mapping, Report no. 1038-96, LaBRI, University of Bordeaux I, 1996.

[PEL 08] PELLEGRINI F., User's guide, ENSEIRB - LaBRI, University of Bordeaux I, scotch 5.1 edition, 2008.

[POT 90] POTHEN A., SIMON H.D., LIOU K.-P., "Partitioning sparse matrices with eigenvectors of graphs", _SIAM Journal on Matrix Analysis and Applications_ , vol. 11, no. 3, p. 430-452, 1990.

[PRE 98] PREIS R., DIEKMANN R., The Party Partitioning Library, User Guide, University of Paderborn, 1.99 edition, October 1998.

[SCH 03] SCHAMBERGER S., "Improvements to the helpful-set heuristic and a new evaluation scheme for graphs-partitioners", in _Proceedings of the International Conference on Computational Science and its Applications_ , Lecture Notes in Computer Sciences, Springer, p. 49-59, 2003.

[SHI 00] SHI J., MALIK J., "Normalized cuts and image segmentation", _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 22, no. 8, p. 888-905, 2000.

[SIM 91] SIMON H.D., "Partitioning of unstructured problems for parallel processing", _Computing Systems in Engineering_ , vol. 2, p. 135-148, 1991.

[WAL 00] WALSHAW C., CROSS M., "Mesh partitioning: a multilevel balancing and refinement algorithm", _SIAM Journal on Scientific Computing_ , vol. 22, p. 63-80, 2000.

[WAL 02] WALSHAW C., The Jostle Executable User Guide, University of Greenwich, 3.1 edition, July 2002.

[WIL 91] WILLIAMS R.D., "Performance of dynamic load balancing algorithms for unstructured mesh calculations", _Concurrency_ , vol. 3, p. 457-481, 1991.

* * *

1 Appendix written by Charles-Edmond BICHOT.

1 Definitions and differences between the constraint and unconstrained graph partitioning optimization problems are presented in Chapter 1.

2 The spectral graph partitioning method is presented in Chapter 2, section 2.6.

3 The spectral method is described in Chapter 2, section 2.6, page 52.

4 The normalized cut objective function is described in Chapter 1, section 1.5, page 11.

5 The ivview software is a part of the Open Inventor toolkit whose Website is: oss.sgi.com/projects/inventor/.

# Glossary

**Acronym** | **Description**
---|---
_bal_ () | Calculation function of the balance of a partition _P k_
CFMU | Central flow management unit
_cut_ () | Cost function of the cut of a graph _G_
DGAC | _Direction générale de l'aviation civile_ (French civil aviation authority)
DRB | Dual recursive bipartitioning
_E_ | Set of edges of a graph _G_
FAB | Functional airspace block
FF | Fusion-fission, graph partitioning algorithm
FL | Flight level
FM | Fiduccia-Mattheyses algorithm
_G_ | Graph, _G_ = ( _V, E_ )
GA | Genetic algorithms
GBA | Genetic bisection algorithm
GGGP | Greedy graph growing partitioning
GGP | Graph growing partitioning
GKLR | Global Kernighan-Lin refinement
GRASP | Greedy randomized adaptive search procedure
GSM | Global system for mobile communications
HEM | Heavy edge matching
ILS | Iterative local search
IMP | Iterative multilevel partitioning
IPP | Protein-protein interaction graph
JE | Jostle evolutionary
KL | Kernighan-Lin algorithm
LU | Decomposition of a matrix into a product of two triangular matrices: **L** ( _Low_ ) and **U** ( _Up_ )
  | Set of natural numbers
 * | Set of non-zero natural numbers
NUMA | Non-uniform memory architecture
_P k_ | Partition of edges of a graph _G_ into _k_ parts
PROBE | Population reinforced optimization based exploration
  | Set of real numbers
SBILS | Simple bisection iterated local search
SHEM | Sorted heavy edge matching
TIG | Task interaction graph
UTC | Coordinated universal time
_V_ | Set of edges of a graph _G_
VLSI | Very-large-scale integration
_w_ () | Weight of an edge or sum of weights of a set of elements

# List of Authors

Jean-Baptiste ANGELELLI
IML
Aix-Marseille University
France

Thomas AYNAUD
LIP6
University Pierre and Marie Curie
Paris
France

Charles-Edmond BICHOT
LIRIS
Ecole centrale de Lyon
University of Lyon
France

Vincent D. BLONDEL
Department of Mathematical Engineering
Catholic University of Louvain
Belgium

Alexandre CAMINADA
SET
UTBM
Belfort-Montbéliard
France

Cédric CHEVALIER
Sandia National Laboratories
Albuquerque, New Mexico
USA

Nicolas DURAND
POM
DSNA/DTI/R&D
Toulouse
France

Alain GUÉNOCHE
IML
CNRS
Marseille
France

Jean-Loup GUILLAUME
LIP6
University Pierre and Marie Curie
Paris
France

Renaud LAMBIOTTE
FUNDP: Naxys
Facultés Universitaires Notre-Dame de la Paix
Namur
Belgium

Sid LAMROUS
SET
UTBM
Belfort-Montbéliard
France

Laurent NAJMAN
LIGM
ESIEE
Noisy le Grand
France

Amir NAKIB
LiSSi
University of Paris-Est Créteil
France

Mustapha OUGHDI
SET
UTBM
Belfort-Montbéliard
France

François PELLEGRINI
INRIA
Bordeaux Sud-Ouest - LaBRI
Institut polytechnique de Bordeaux
France

Laurence REBOUL
Mathematics and Applications Laboratory
University of Poitiers
France

Patrick SIARRY
LiSSi
University Paris-Est Créteil
France

Hugues TALBOT
LIGM
ESIEE
Noisy le Grand
France

# Index

**A**

air traffic

control sector

functional airspace block

objective

qualification zone

algorithm

Bokhari

Fiduccia-Mattheyses

genetic bisection algorithm

Gibbs-Poole-Stockmeyer

graph growing partitioning

greedy graph growing partitioning

iterated multilevel partitioning

Jostle evolutionary

Kernighan-Lin

_k_ -means

aspect ratio

**B**

bisection

recursive

**C**

cell

dimensioning

centered clique

class system

coarsening

heavy edge matching

Hendrickson-Leland

sorted heavy edge matching

communities

contraction

heavy connectivity matching

inner product matching

corrected Rand index

cost function

communication

cut

inertia

modularity

normalized

partial

cut λ − 1

**D**

demand profile

dendrogram

diameter

dilation

distance

Euclidean

Hamming

Manhattan

transfer

**F**

fusion-fission

**G**

graph

adjacency

artificial

band

bipartite

coarsening, see multilevel

cocycle

coloring

density

distributed

flow maximum

protein-protein interaction

quotient

random

real

shortest path

**H**

halo

hierarchical method

hierarchy

hyperedges

hypergraph

column-net

partitioning

row-net

**I**

image

contour detection

denoising

markers

restoration

segmentation

deformable model

power watersheds

stereovision

**L**

linear systems resolution

logit

**M**

matching

matrix

adjacency

Laplacian

metaheuristics

ant colonies

genetic algorithms

iterated local search

probe

simulated annealing

modularity

optimization

resolution limit

multilevel

coarsening

expansion

folding

growing

partitioning

refinement

uncoarsening

mutual information

**N**

net

non-supervised classification

**P**

parallel calculation

parallel computing

partition

comparaison

partitioning

balance

constrained

unconstrained

partitioning tools

Chaco

Graclus

Hmetis

Jostle

Metis

Parmetis

Party

Parjostle

Parkway

Patoh

Pt-Scotch

Scotch

Zoltan

Pearson correlation

pin

**R**

random walk

refinement

Walshaw-Cross

region growing

reordering

local

repartitioning

routing

**S**

scalability

simulation protocol

spectral

static mapping

partial

**T**

tarification

test benches

artificial graphs

community detection

graph partition archive,

**V**

vertex

ghost, halo

peripheral

pseudo-peripheral

**Z**

Zoltan

redpajama_set_name: RedPajamaBook