Date: Wednesday, July 22, 2009
Time: 2-4pm
Place: Volen 101

The Complexity of Flow Analysis in Higher-Order Languages
David Van Horn

Abstract:

This dissertation proves lower bounds on the inherent difficulty of deciding flow analysis problems in higher-order programming languages. We give exact characterizations of the computational complexity of 0CFA, the kCFA hierarchy, and related analyses. In each case, we precisely capture both the expressiveness and feasibility of the analysis, identifying the elements responsible for the trade-off.

0CFA is complete for polynomial time. This result relies on the insight that when a program is linear (each bound variable occurs exactly once), the analysis makes no approximation; abstract and concrete interpretation coincide, and therefore program analysis becomes evaluation under another guise. Moreover, this is true not only for 0CFA, but for a number of further approximations to 0CFA. In each case, we derive polynomial time completeness results.

For any k > 0, kCFA is complete for exponential time. Even when k = 1, the distinction in binding contexts results in a limited form of closures, which do not occur in 0CFA. This theorem validates empirical observations that kCFA is intractably slow for any k > 0. There is, in the worst case—and plausibly, in practice—no way to tame the cost of the analysis. Exponential time is required. The empirically observed intractability of this analysis can be understood as being inherent in the approximation problem being solved, rather than reflecting unfortunate gaps in our programming abilities.

Linearity has been the key feature in several lines of research in both theoretical and practical approaches to computer science. In the theoretical side all the work stemming from linear logic dealing with proof technology, complexity classes and more recently quantum computation. In the practical side work on program analysis, expressive operational semantics for programming languages, linear programming languages, program transformation, update analysis and efficient implementation techniques. The aim of this workshop is to bring together researchers who are currently developing theory and applications of linear calculi, in order to foster their interaction, to provide a forum for presenting new ideas and work in progress, and to enable newcomers to learn about current activities in this area. LINEARITY 2009 will be a one-day satellite event of CSL 2009.

Flow analysis is a ubiquitous and much-studied component of compiler technology—and its variations abound. Amongst the most well known is Shivers’ 0CFA; however, the best known algorithm for 0CFA requires time cubic in the size of the analyzed program and is unlikely to be improved. Consequently, several analyses have been designed to approximate 0CFA by trading precision for faster computation. Henglein’s simple closure analysis, for example, forfeits the notion of directionality in flows and enjoys an “almost linear” time algorithm. But in making trade-offs between precision and complexity, what has been given up and what has been gained? Where do these analyses differ and where do they coincide?

We identify a core language—the linear λ-calculus—where 0CFA, simple closure analysis, and many other known approximations or restrictions to 0CFA are rendered identical. Moreover, for this core language, analysis corresponds with (instrumented) evaluation. Because analysis faithfully captures evaluation, and because the linear λ-calculus is complete for PTIME, we derive PTIME-completeness results for all of these analyses.

See you in Valencia!

Comments are welcomed and appreciated. This has been submitted to SAS, but there is still plenty of time to revise.

First, we’ll rely on Church’s original encoding of pairs. Here is the constructor for pairs; it’s like a curried version of cons, where cons cells are represented with procedures:

(define pair (λ (x) (λ (y) (λ (z) ((z x) y)))))

To destructure a pair p, you apply it to a procedure like:

(p (λ (x) (λ (y) ... x ... y ...)))

Where x and y will refer to the car and cdr of the pair p, respectively. We can make this look a little nicer with the following syntactic sugar:

(define-syntax let-pair
  (syntax-rules ()
    ((let-pair ((x y) p) e)
     (p (λ (x) (λ (y) e))))))

Now we add a little bit more sugar for functions on pairs:

(define-syntax λ*
  (syntax-rules ()
    ((λ* (x y) e)
     (λ (p) (let-pair ((x y) p) e)))))

The Booleans are built out of the twist and identity functions on pairs:

(define tt (λ* (x y) ((pair x) y)))
(define ff (λ* (x y) ((pair y) x)))

The idea is that Boolean values (pairs) carry their “real” value in the first component, and their negation in the second component. So we define True and False as follows:

(define true  ((pair tt) ff))
(define false ((pair ff) tt))

The Not connective is easy, it is simply an inversion:

(define not (λ* (u v) ((pair v) u)))

And is not so easy. Let’s build it up in pieces. And takes a pair of Boolean values, each of which is a pair of functions:

(define and
  (λ* (a b)
    (let-pair ((p^ p*) a)
    (let-pair ((q^ q*) b)
       ...))))

Each of the ps and qs are functions on pairs—either twist or identity—and we have the “real” value in the first component and the negation in the second component. So if a is True, then p^ is identity, and twist otherwise. If we apply p^ to the pair (q^,ff), we’ll get (q^,ff) when a is True and (ff,q^), otherwise. Looking only at the first component of this pair gives us the answer for computing the And of a and b, and we can read it much like the usual reading of Church’s encodings of the Booleans: “If p^, then q^, else false (aka, ff).”

But now what do we do with the negative components of a and b? We just apply the DeMoorgan dual, namely Or, which reads: “If p*, then true (aka, tt), else q*.” So far we have:

(define and
  (λ* (a b)
    (let-pair ((p^ p*) a)
    (let-pair ((q^ q*) b)
    (let-pair ((u^ v^) (p^ ((pair q^) ff)))
    (let-pair ((u* v*) (p* ((pair tt) q*)))
      ...))))))

We’d like to able to return the pair (u^,u*), ie. the first component of computing the And of the first components of a and b, and the first component of computing the Or of their negation. But this wouldn’t be linear—we’re throwing away the “junk” v^,v*.

But we know something important about this junk, namely it’s symmetric. If v^ is tt, then v* is ff, and if v^ is ff, then v* is tt. One is twist, one is identity, we just don’t know which is which. But we don’t need to. If we compose v^ and v* we’re guaranteed to get twist. Compose this with another twist (ff) and we get identity. Compose this with v* and we get v*—but now we’ve used everything and maintained linearity! So we have:

(define and
  (λ* (a b)
    (let-pair ((p^ p*) a)
    (let-pair ((q^ q*) b)
    (let-pair ((u^ v^) (p^ ((pair q^) ff)))
    (let-pair ((u* v*) (p* ((pair tt) q*)))
      ((pair u^)
       (compose ((pair u*)
                 (compose ((pair v^)
                           (compose ((pair v*) ff)))))))))))))

Where composition is defined as:

(define compose (λ* (f g) (λ (x) (f (g x)))))

The slogan is: “Symmetric garbage is self-annihilating”, and we can build up other logical connectives, like Or, Implies, etc. in exactly this same way.

But what about Boolean formulas that are non-linear, ie. variables occur more than once? For example, how can we write “b or not b”, which is a tautology? We need an explicit Copy connective that will “fan out” a Boolean value. It seems counter intuitive that we can even write a Copy operation in a linear way. After all, linearity is all about NOT being able to copy things. But we’ll see that it can be done quite easily in this encoding of the Booleans. Since any Boolean value is a pair of functions on pairs, how can we copy a value? We just apply these functions to pairs of functions on pairs. For example, tt applied to (tt,ff) is (tt,ff), which is the encoding of True. Applying ff to (tt,ff) is (ff,tt), which is the encoding of False. So if you take a Boolean and apply it’s first component to True, you get back True whenever the Boolean is True and False whenever the Boolean is False. What about the second component? Applying it to False gives you True whenever the Boolean is True (the Boolean has twist in the second component so it takes False to True) and False whenever the Boolean is False. So if you apply the first component to True and the second component to False, what do you get back? A pair of Trues or a pair of Falses depending on the value of the Boolean, ie. two copies! So the Copy connective is:

(define copy (λ* (p q) ((pair (p true)) (q false))))

And now we’re done. We’ve given a nice readable encoding of the Booleans in linear λ-calculus using Scheme—just as readable as the linear ML counterpart. And if you were to expand away the syntactic sugar and inline the definitions of the connectives, you’d be left with a Scheme program that uses only applies and λs (taking exactly one argument)—no pattern matching or tuples needed.

• (λf.f f (λy.y)) (λx.x)

This program has an exact 1CFA. But perform the following linear expansion:

• apply (λf.f f (λy.y)) (λx.x)

Where apply ≡λxλy.xy, then 1CFA is no longer exact. So inserting a linear term can introduce approximation, whereas this is not the case in 0CFA, and really it shouldn’t be the case in kCFA. We need a variant of kCFA that recognizes linear (virtual) reductions and performs them; there clearly is no harm in doing so (the code size cannot blow up), and there is a precision gained at no cost.

I noticed this because these linear expansions are at the heart of two proofs appearing at ICFP this year. One is mine, where I show that the NP-hardness construction for 1CFA can be adapted to kCFA by inserting linear terms that effectively eat the added precision of going from k to (k+1)CFA. Damien Sereni uses a similar trick—with a nicer presentation, by the way—in proving a separation result about k- and (k+1)-limited CFA.

If you had an analysis that was invariant under linear expansion, these proofs wouldn’t hold, and moreover, the theorems might not be true! But beyond aggravating theorists, this is clearly the right thing to do from a pragmatic point of view: why pay for lunch if you’re not going to eat it?

• [2007,inproceedings] bibtex
  D. Sereni, "Termination Analysis and Call Graph Construction for Higher-Order Functional Programs," in Proceedings of the 12th ACM SIGPLAN International Conference on Functional Programming (ICFP 2007), 2007.
  @INPROCEEDINGS{sereni_icfp07,
  author = {Damien Sereni},
  title = {Termination Analysis and Call Graph Construction for Higher-Order Functional Programs},
  booktitle = {Proceedings of the 12th ACM SIGPLAN International Conference on Functional Programming (ICFP 2007)},
  editor = {Norman Ramsey},
  publisher = {ACM Press},
  year = {2007},
  url = {http://metacomp.comlab.ox.ac.uk/Members/damien/publications/icfp07.pdf},
  }

• [2007,inproceedings] bibtex
  D. Van Horn and H. G. Mairson, "Relating Complexity and Precision in Control Flow Analysis," in Proceedings of the 2007 ACM SIGPLAN International Conference on Functional Programming (ICFP 2007), New York, NY, USA, 2007, pp. 85-96.
  @INPROCEEDINGS{vanhorn_icfp07,
  author = {David {Van Horn} and Harry G. Mairson},
  title = {Relating Complexity and Precision in Control Flow Analysis},
  booktitle = {Proceedings of the 2007 ACM SIGPLAN International Conference on Functional Programming (ICFP 2007)},
  year = {2007},
  isbn = {978-1-59593-815-2},
  pages = {85--96},
  location = {Freiburg, Germany},
  doi = {http://doi.acm.org/10.1145/1291151.1291166},
  publisher = {ACM Press},
  address = {New York, NY, USA},
  url = {http://www.cs.brandeis.edu/~dvanhorn/pubs/vanhorn-mairson-icfp07.pdf}
}