Doctor JS is a genius. He’ll analyze your JavaScript code, complete with: polymorphism, prototypes, exceptions, and callbacks.

DoctorJS is a static analysis tool developed by my PRL lab neighbor, Dimitris Vardoulakis. It’s an implementation of Vardoulakis and Olin Shivers‘ CFA2—a context-free approach to control flow analysis.

Brendan Eich, CTO of Mozilla and creator of JS, recently blogged about DoctorJS (Static Analysis FTW) and called static analysis one of the best “researchy” investments Mozilla has made over the past few years.

Abstracting Abstract Machines.
David Van Horn and Matthew Might.
The 15th ACM SIGPLAN International Conference on Functional Programming (ICFP’10), Baltimore, Maryland, September, 2010.

We describe a derivational approach to abstract interpretation that yields novel and transparently sound static analyses when applied to well-established abstract machines. To demonstrate the technique and support our claim, we transform the CEK machine of Felleisen and Friedman, a lazy variant of Krivine’s machine, and the stack-inspecting CM machine of Clements and Felleisen into abstract interpretations of themselves. The resulting analyses bound temporal ordering of program events; predict return-flow and stack-inspection behavior; and approximate the flow and evaluation of by-need parameters. For all of these machines, we find that a series of well-known concrete machine refactorings, plus a technique we call store-allocated continuations, leads to machines that abstract into static analyses simply by bounding their stores. We demonstrate that the technique scales up uniformly to allow static analysis of realistic language features, including tail calls, conditionals, side effects, exceptions, first-class continuations, and even garbage collection.

• PDF

The second technique keeps continuations on the stack to achieve a push-down model of abstract interpretation. The resulting abstract interpreter always matches calls and returns, achieving a higher level of precision by never conflating call and return pairs. Although this technique produces abstract interpreters with infinite state-spaces, we demonstrate how basic static analysis questions remain decidable by casting them as language inclusion problems answered by push-down automata.

Pushdown Control-Flow Analysis of Higher-Order Programs.
Christopher Earl, Matthew Might, and David Van Horn.
The 2010 Workshop on Scheme and Functional Programming (SFP 2010), Montréal, Québec, Canada, August, 2010.

Context-free approaches to static analysis gain precision over classical approaches by perfectly matching returns to call sites—a property that eliminates spurious interprocedural paths. Vardoulakis and Shivers’s recent formulation of CFA2 showed that it is possible (if expensive) to apply context-free methods to higher-order languages and gain the same boost in precision achieved over first-order programs.

To this young body of work on context-free analysis of higher-order programs, we contribute a pushdown control-flow analysis framework, which we derive as an abstract interpretation of a CESK machine with an unbounded stack. One instantiation of this framework marks the first polyvariant pushdown analysis of higher-order programs; another marks the first polynomial-time analysis. In the end, we arrive at a framework for control-flow analysis that can efficiently compute pushdown generalizations of classical control-flow analyses.

• PDF

There are also slides (PDF) covering both techniques from a talk given at the Harvard PL seminar.

• Preprint
• Slides from NEPLS
• Implementation

Abstract:

Low-level program analysis is a fundamental problem, taking the shape of “flow analysis” in functional languages and “points-to” analysis in imperative and object-oriented (OO) languages. Despite the similarities, the vocabulary and results in the two communities remain largely distinct, with limited cross-understanding. One of the few links is Shivers’s k-CFA work, which has advanced the concept of “context-sensitive analysis” and is widely known in both communities. Recent results, however, indicate that the relationship between the different incarnations of the analysis is not understood. Van Horn and Mairson proved k-CFA for k ≥ 1 to be EXPTIME-complete, hence no polynomial algorithm exists. Yet there have been multiple polynomial formulations of context-sensitive points-to analyses in OO languages. Is functional k-CFA a profoundly different analysis from OO k-CFA? We resolve this paradox by showing that OO features conspire to make the exact same specification of k-CFA be polynomial-time: objects and closures are subtly different, in a way that interacts crucially with context-sensitivity. This leads to a significant practical result: by emulating the OO approximation, we derive a polynomial hierarchy of context-sensitive CFAs for functional programs, simultaneously achieving high precision and efficiency.

Update (2/2010): This work will appear at PLDI 2010. See you in Toronto!

Update (2/2010): Noticed on LtU: http://lambda-the-ultimate.org/node/3814.

Update (3/2010): Camera-ready version: http://www.ccs.neu.edu/home/dvanhorn/pubs/might-smaragdakis-vanhorn-pldi10.pdf.

The HOFA forum is an email forum for the discussion and dissemination of research results in the area of higher-order flow analysis, broadly construed, within computer science and related disciplines. Flow analysis and related static analyses are a fundamental tool for program verification, bug detection, compiler optimization, program understanding, and software maintenance. The HOFA forum aims to facilitate theoretical, practical, and application advances in the area of functional, object-oriented, concurrent, distributed, and mobile programming.

Please sign up! Also, send me email if you or your group would like to be listed on the HOFA site.

Date: Wednesday, July 22, 2009
Time: 2-4pm
Place: Volen 101

The Complexity of Flow Analysis in Higher-Order Languages
David Van Horn

Abstract:

This dissertation proves lower bounds on the inherent difficulty of deciding flow analysis problems in higher-order programming languages. We give exact characterizations of the computational complexity of 0CFA, the kCFA hierarchy, and related analyses. In each case, we precisely capture both the expressiveness and feasibility of the analysis, identifying the elements responsible for the trade-off.

0CFA is complete for polynomial time. This result relies on the insight that when a program is linear (each bound variable occurs exactly once), the analysis makes no approximation; abstract and concrete interpretation coincide, and therefore program analysis becomes evaluation under another guise. Moreover, this is true not only for 0CFA, but for a number of further approximations to 0CFA. In each case, we derive polynomial time completeness results.

For any k > 0, kCFA is complete for exponential time. Even when k = 1, the distinction in binding contexts results in a limited form of closures, which do not occur in 0CFA. This theorem validates empirical observations that kCFA is intractably slow for any k > 0. There is, in the worst case—and plausibly, in practice—no way to tame the cost of the analysis. Exponential time is required. The empirically observed intractability of this analysis can be understood as being inherent in the approximation problem being solved, rather than reflecting unfortunate gaps in our programming abilities.

We give the first direct subcubic algorithm for performing control flow analysis of higher-order functional programs. Despite the long held belief that inclusion-based flow analysis could not surpass the “cubic bottleneck, ” we apply known set compression techniques to obtain an algorithm that runs in time O(n^3/log n) on a unit cost random-access memory model machine. Moreover, we refine the initial flow analysis into two more precise analyses incorporating notions of reachability. We give subcubic algorithms for these more precise analyses and relate them to an existing analysis from the literature.

Flow analysis is a ubiquitous and much-studied component of compiler technology—and its variations abound. Amongst the most well known is Shivers’ 0CFA; however, the best known algorithm for 0CFA requires time cubic in the size of the analyzed program and is unlikely to be improved. Consequently, several analyses have been designed to approximate 0CFA by trading precision for faster computation. Henglein’s simple closure analysis, for example, forfeits the notion of directionality in flows and enjoys an “almost linear” time algorithm. But in making trade-offs between precision and complexity, what has been given up and what has been gained? Where do these analyses differ and where do they coincide?

We identify a core language—the linear λ-calculus—where 0CFA, simple closure analysis, and many other known approximations or restrictions to 0CFA are rendered identical. Moreover, for this core language, analysis corresponds with (instrumented) evaluation. Because analysis faithfully captures evaluation, and because the linear λ-calculus is complete for PTIME, we derive PTIME-completeness results for all of these analyses.

See you in Valencia!

We give an exact characterization of the computational complexity of the kCFA hierarchy. For any k > 0, we prove that the control flow decision problem is complete for deterministic exponential time. This theorem validates empirical observations that such control flow analysis is intractable. It also provides more general insight into the complexity of abstract interpretation.

Comments welcome and appreciated.