Hints for Infrastructure Developers (Reuse for Computer Science Software Research)

David (Pardo) Keppel pardo@xsim.com

Michael D. Smith smith@eecs.harvard.edu

Todd Austin austin@umich.edu

September 2000; revised March 2003 and April 2004

Abstract

An infrastructure provides a resource which can be used widely. Done well, an infrastructure can enable new and better research. An especially successful infrastructure can become a standard, improving comparisons betweens systems and raising the standard for subsequent research. However, it is suprisingly difficult to produce successful infrastructure. This paper surveys several successful systems and gives recommendations learned from these earlier systems.

This note gives hints on how to build works of code, data, and/or documentation that are prepared by one group and used by other groups to perform computer science research. This note, in the spirit of Butler Lampson’s “Hints for Computer System Design” [Lam84], presents these hints as rules of thumb to consider while producing infrastructure. These hints are not always appropriate or necessary; but simply pausing to reflect on which ones are right for your task will give you a head-start on producing a successful infrastructure.

The primary goal of an infrastructure is to provide a resource that you and others can use and share. If done well, your infrastructure enables accurate and meaningful comparisons of competing approaches within a design space. Or, it reduces the cost of entry into a research domain by allowing others to explore or implement what is new and reuse what exists or is already well known.

A particularly successful infrastructure can provide a research standard that other works in the same area must meet – it can do for experimental system design what SPEC [SPEC] has done for benchmarking. And, distributing your infrastructure may give you access to a communicty of developers that help you to improve and expand your infrastructure beyond its initial intended purposes and accelerate the pace of progress in your field.

We have discovered, through trial and error, that it is difficult to produce useful infrastructure, where useful is defined to mean that the infrastructure meets one or more of the goals stated above. This is at first surprising, since the essence of software is reuse and ease of copying. Several factors, however, make it difficult to provide a bucket of bytes that others will actually use. Experience appears to be a big factor in developing infrastructure. We hope this note helps you to avoid some of our mistakes.

1 Our process and the process

Our process that generated this note was ad-hoc. We simply studied several infrastructure projects we considered to be successful and looked for similarities. We discussed both successful choices and also important mistakes from which lessons were learned. Our study was biased towards reusable code, but some of the rules apply to non-code entities.

During our review, one point rose above all others: Building infrastructure is a process, rather than a simple thing. The process involves much more than simply writing code or collating results. In particular, it differs much from most coding in programming classes. A bug in your software no longer means just a few points off; it often means that others will find your infrastructure unusable. Users will have questions, and you have to decide how these questions are answered. Fixing bugs, answering questions, and maintaining distribution channels all have costs, and you must decide who will perform the labor and provide the money.

Thus, while the infrastructure itself is technical in nature, the overall process has many facets. We identified common themes along four general dimensions: economic, education, sociology (or maybe psychology), and technology. Our list includes hints in each of these, and some hints touch on more than one dimension. Apply as needed.

2 Hints

We started by considering a variety of infrastructures. This section summarizes our results as a set of rules. Section 3 considers the infrastructures in detail and considers how some of our rules apply.

2.1 Important non-technical costs

Consider funding: It typically costs money to develop infrastructure, but lifetime costs can be even larger. Lifetime costs include finding and fixing bugs, answering user questions about installation, preparing new releases, etc. Lifetime costs are especially high since support work is often boring compared to initial development and thus there are fewer volunteers. Also, infrastructure may survive long after the last developer is gone, so there may be nobody familiar with the code. Be prepared to keep funding the project along the way, and be prepared to drop support when the money vanishes.

Be a champion: Projects are often more successful if there is an identifiable individual or group that has an interest in the basic problem addressed by the infrastructure. A champion can help to centralize the design decisions and direct the integration of new pieces. Furthermore, an infrastructure is much more likely to be accepted by a community if there is a promise of continued development and maintenance by the champion.

Advertise: Make sure that information is widely disseminated about the infrastructure. The goal is to ensure that, over time, people who can make use of the infrastructure will be able to find it. “Advertise” does not necessarily mean spending money. Do try to get listed in relevant reviews/summaries, post to relevant discussion lists, and so on. Encourage users to sign up for a low-traffic announcements list.

2.2 Background research

Identify the receptive community: One part of infrastructure success is getting a critical mass of users. A good way to develop critical mass is to identify and support the portion of the community which most desperately needs the infrastructure. If you use an open source model, you should use programming languages and tools that are widely accepted by that community.

Compete: It is useful to describe why an infrastructure is competitive with or better than other similar infrastructures, so that potential users can quickly understand the relative advantages and disadvantages of each.

2.3 Legal restrictions

Simplify legal restrictions: The rules governing use of the infrastructure should be simple and clear, in order to avoid scaring away potential users and having others use your infrastructure in ways that you did not wish, for example commercial uses. In addition, it should be simple to perform licensing. For example, “point and click” agreements are superior to mail-in forms. If you do have restrictions in your license, consult a qualified IP attorney and file trademark and copyright paperwork to protect your rights.

Be careful with user-contributed code: User-contributed code may have hidden or tricky legal restrictions, depending on the origins, especially if your infrastructure has legal restrictions. However, you may wish to provide a repository of user-supplied code which other users can apply and use on their own. Keeping the contributed code separate simplifies legal encumbrance. Very simple bug fixes are likely free from legal complexities.

2.4 Encouraging wide-spread use

Make it free to try and cheap to use: People shy away from infrastructure that is expensive to try or use. The lower the demo cost, the more likely it is that people will try out the infrastructure and therefore get a realistic view of its capabilities. The lower the use cost, the more likely that people will consider using the infrastructure in a project.

Make it easy to get started: People shy away from infrastructure that is difficult to “test drive” or hard to use for simple tasks. An infrastructure should come with a variety of aids that make it easy to get started. Aids include: tutorial introduction; simple examples; and as simple a build/configure/setup procedure as can be managed.

Encourage educational use: Educational users have little money but may be especially supportive and willing to provide feedback, bug fixes, etc. Educational exposure often leads to industrial exposure as students graduate and move to industry.

Consider multiple distribution formats: For binary data, consider both binary and text forms, since different systems have different binary representations. For executable tools, consider both source code and pre-built linkable object or binary distributions. Users with compatible systems can download and use the infrastructure without configuration, conversion, etc.; while those without compatible systems still have an opportunity to use the infrastructure. In comparison, a single distribution format is typically the format which is the lowest common denominator, and it thus requires all users to go through the more complex setup before they can even try the infrastructure.

Make it portable: An infrastructure is more appealing and more useful if it works on more systems. Make portability an explicit goal, and expect platform-dependent failures. For example, provide regression tests which will run as part of the build, so that “obvious” failures will be caught before the user tries to use or blame the infrastructure. Educational users, and some commercial users, have little choice in the type of system they use, and so both kinds of users benefit from portability.

Make it easy to set up: An important part of infrastructure is making sure that a large number of people can easily try it to see if it meets their needs. Non-trivial setup is a suprisingly large barrier to getting people to try infrastructure for the first time. Easy setup encourages potential users to try the infrastructure; and existing users will mention easy setup when recommending the infrastructure to others. As an added benefit, a simple and clean set up will dramatically reduce the amount of support needed by new users.

Use familiar tools and languages: Potential users will be concerned about compatibility even before they demo an infrastructure. Many potential users will be discouraged if they must first get and install third-party languages or packages just to try a demo. If they do demo the infrastructure, unfamiliar tools, languages, linking rules, user interfaces, and so on can require the users to learn several new systems at once and thus make a simple system appear cumbersome. Familiar tools and languages encourage a user community to help develop and maintain your infrastructure.

Establish a user community: Once the infrastructure starts to catch on, create a place where users can come to interact and contribute to the project. Web sites, mailing lists, and news groups are all great ways to bring together a community of users. An established user community helps to attract new users and also provides substantial support.

2.5 Writing anciliary text

Document: Good documentation is more than some comments in your code or a README file in the distribution. Your distribution should include at least: a tutorial document, working examples (a cookbook), and a detailed manual, which need not be tutorial. Be aware that simply shipping one document containing all of the details of your infrastructure may overwhelm users, who then ask “what part of the documentation must I read?” Sometimes it is good to provide separate documentation for each class of users.

Provide examples: Examples serve several purposes. Examples demonstrate the abilities of the infrastructure; they demonstrate that it even works; and they provide a complete and working starting point upon which the user can build. A good set of examples should begin with the smallest possible example — the equivalent of a C program “Hello, world.” — and include a variety of other realistic examples that exhibit a range of infrastructure capabilities without being entirely trivial.

2.6 Bugs

Make it reliable: Your infrastructure needs to be reliable enough that the user has high confidence in the results. Poor reliability is also a barrier to use. If the infrastructure fails often or in mysterious ways, users are unlikely to adopt it.

Provide tests: Basic functionality tests help in several ways. A test suite helps find bugs more quickly. Development and debugging are faster because you pinpoint and fix bugs, instead of sending them to end users; and your infrastructure is less flaky. Bugs often arise through interaction of subtle things, so similar or recurring bugs are common. Thus, for each bug, add a regression test. The test suite should be distributed with the infrastructure and run as part of the build process in order to help ensure that it runs on the user’s machine as well as on your machine.

Make it robust against user errors: The infrastructure needs to be robust enough that the user spends relatively little time dealing with mysterious failures. The more mysterious the failures, the more likely users will contact you for help or stop using the infrastructure. Robustness against user errors also makes it easier for novice users to explore and use the system by hacking the provided examples. That in turn improves ease-of-use and reduces the need for documentation.

Encourage community debugging: Expose debug facilities to users so that the community can help you locate and fix bugs. When they find your bugs, it helps to improve the infrastructure; when they find their own bugs, it helps cut down on your support time.

Provide alpha and beta releases: Give power users early access to new developments. They often are able to find and fix problems with amazing deft. Power users are often the best advocates for an infrastructure and may help to field user questions.

2.7 Interfaces

Design the interfaces: Infrastructure users are most concerned with using the infrastructure and not on the internal details. Therefore, spend extra effort designing the interfaces to satisfy a wide range of users. It may be a win to slightly complicate the implementation in order to simplify the interface. It may be desirable to provide a mechanism to initialize the infrastructure with one or more sets of defaults. It may be useful to provide a multi-level interface, with a simple form for basic operations and a more complex form for users who require more precise control.

Do what is needed, not more: It is often tempting to provide extra features. Resist the urge. More features mean more corner cases and bugs, more documentation, a more complicated interface, and so on. Instead, do what is needed and do more later if it seems justified.

Generalize: This is in partial conflict with the above, but is worth considering anyway. Consider what problem the infrastructure solves and deliberate whether a more general solution is possible and whether it would make the infrastructure a lot more useful. Note that generalizing is different than adding features. For example, it may be desirable to strip away parts of a tool and export only the core, in order to make the general-purpose part available without burdening the user with domain-specific parts of the original tool.

Read and write an ASCII form: The infrastructure may also have APIs, etc., but it is useful to be able to communicate along both official and unofficial interfaces using plain text. For example, compiler infrastructure which provides a set of optimizations might have a facility to read and write the compiler IR between each optimization phase. The plain text form is useful for several reasons. It helps the user understand what the tool is doing, and thus helps with documentation and debugging. It also provides another interface by which people can use part but not all of an infrastructure. It is also an interface by which they can extend the infrastructure – by reading, modifying, then reinserting the modified text. Worry about functionality which is available through one interface but not the other.

Make it extensible: Consider an infrastructure design that is extensible, with hooks or callbacks for user-supplied code. Be sure to use your own extensibility mechanisms. This will serve as examples for your user population and verify for you that the extensibility mechanisms are useful.

Build on other infrastructures: The goal of infrastructure is to reduce work, so build your own infrastructure on existing infrastructure. Plan for changes in that infrastructure. For example, consider creating a wrapper interface between your infrastructure and the client infrastructure that it calls. By making explicit abstractions of the clients, you can both reduce maintainance overheads as the client changes, and also make it easier to port your infrastructure to run on other similar kinds of client infrastructures.

2.8 Improving your infrastructure

Clean up the implementation: An infrastructure is best if it starts with a clean design and implementation. However, many systems are grown rather than being designed, and even deisgned systems will grow over time. Cleanups help make the infrastructure more reliable, robust against internal errors, robust against user errors, and so on, and also make it easier for users to diagnose and solve errors. For example, check parameters for legality, check for NULL pointers, and call a single uniform routine to display error messages. A clean implementation will be easier to maintain when you come back two years later to fix a bug. Clean code is also useful for users who read the code. Code can serve as further documentation. Users may want to modify the infrastructure to exactly meet their needs. Note that code cleanup is an ongoing process, as the interface, implementation, and so on change over time.

Evolve: Many infrastructures arise from tools; and many systems evolve. Be on the lookout for opportunities to evolve the infrastructure on all of of these dimensions: distribution mechanisms can be simplified; documentation can be improved; interfaces can be evolved; and so on. Support calls and user feedback can provide many useful clues about how to change the infrastructure both to improve its operation and reduce the overhead of maintaining it.

Listen to users: Understand how current infrastructure limits their work. Implement (or incorporate) their suggested improvements.

Build a contribution mechanism: Encourage users to improve the infrastructure. Provide a basic set of tests that must pass before you will accept a contribution from your community of users. Separate that code from the code you distribute so the community knows where to direct their questions. Consider IP issues if you are interested in future commercialization.

Avoid perfection: When an infrastructure serves its intended needs, release it as an alpha release. Leverage user feedback to drive refinement, and continue to release improvemed versions of the system. Be tenacious about improving your work over time, but avoid perfection in the first or even Nth release.

Plan to throw several away: As an infrastructure develops, many components will turn out to be insufficient, or will simply suffer from too much editing and too little redesign. Be prepared to periodically discard and rewrite portions of the infrastructure. At times, you may need to throw the entire infrastructure away. In these cases, avoid sacrificing capability for compatibility.

3 Example Infrastructure Projects

This section discusses several infrastructure projects and how our rules apply. These projects are successful infrastructure in that they are widely cited in the literature as influential; are widely cited as infrastructure components of somebody else’s proejct; we believe the projects are used more widely than reported in the literature; has had a lifetime of a decade or more; and that the infrstructure has been used in ways that surpassed the expectations of the infrastructure designers.

The goal of the following is to discuss our rules in the context of specific infrastructure projects. We chose the projects based on our own familiarity with them and because we did not follow the same rules each time and the variations are interesting. To aid in the comparison, the following text is tagged with the same text as the preceding section headings.

3.1 Shade

Shade is a instruction set execution tracing tool which executes SPARC Solaris binaries and generates a wide variety of trace information. Implementation and useage details are discussed elsewhere [CK94].

Important non-Technical Costs: Shade development, distribution, and maintainence were all funded by Sun Microsystems. It was originally an in-house project that was subsequently released for general use. Sun has supported Shade for nearly fifteen years, about two-thirds of that without involvement of the original authors. The champion for the external community was Sun Microsystems. Advertisement was largely confined to a few published technical papers and word-of-mouth. Word-of-mouth was successful in part because Shade was free, reliable, came recommended by the user community, and because early-on there were no comparable tools.

Background Reasearch: The receptive community was knocking at the door: Shade was developed in response to shortcomings in the available tools. Competition, and the advantages and weaknesses of Shade, were spelled out in Shade technical papers, which provide extensive comparisons of Shade against other simulation and tracing tools. The technical papers are themselves now research tools.

Legal Restrictions: Shade was initially made available under a special license that required signing and sending in a form then waiting for distribution media. It was available only for non-commercial use. More recently Shade was made available under a common license used by Sun Microsystems for many packages, with a point-and-click license and immediate download. Although there have been many user suggestions, distributing object code meant there was little user-contributed code.

Encouraging Wide-Spread Use: Shade is distributed free. It was mainly supplied in object/binary form, along with sources, binaries, and examples for several examples. Getting started could be as simple as unpacking the distribution and typing a single command. Shade was mostly sent to educational users and Sun Microsystems customers. Shade was available on several distribution media (tapes, etc.) but early on was only available in binary format. Lack of source might appear to limit portability, but the implementation of Shade limits it to run only on SPARC/Solaris systems. The lack of portability undoubtedly hurt Shade’s widespread use, but fit the needs of Sun, who was paying for it, and was acceptable for many users. The focus on a single platform helped other issues including distribution formats, interface languages, reliability, etc. Shade uses C as an extension language. Doing so hurts performance but simplifies the implementation, improves reliability, and makes it easier for users to write extensions. Although there were many users, there was relativlely little user community. Factors included lack of flexibility due to binary-only distribution, easy setup and use, good reliability, and lack of competing tools. However, Shade and the users likely would have benefitted from users being able to exchange usage tips and tools built on Shade.

Writing Ancilary Text: Shade is distributed with both a comprehensive tutorial and manual pages. There are also several technical reports, etc., that help power users understand the operation and limitations of Shade. Shade comes with usage examples including manual page examples for provided analyzers; tutorial examples of using built-in tracers; and source code for several analyzers.

Bugs: Shade was designed to be reliable instead of being as fast as possible. The design helped both in preventing bugs and in analyzing failures. Most bugs were found by users within Sun; there were very few bug reports by external users. No tests are provided with Shade, but since it runs on a single family of machines, there is less need for tests. Also, example analyzers and documentation help to avoid usage problems that look like bugs. The Shade interface checks for a variety of user errors. In addition the interface shields the user from many details, reducing the opportunity for errors. For example, the user supplies the size of each trace record and the size of the trace buffer; Shade does calculations on behalf of all user callbacks in order to reduce the chance of user-level errors. There was no community debugging of Shade. We were fortunate the bug rates were low; had it been even slightly more buggy, debugging efforts could have overwhelmed the project, especially in the absense of community debugging. The Shade community within Sun suggested features, served as alpha testers, and ran vast workloads. External releases were therefore mature.

Interfaces: The interface was designed to be easy to use, and ease of use was chosen over peak performance. The design relied in part on past experience with program tracing tools. It also “grew” based on suggestions from users. Because the early user base was small, it was possible to change the interface during development. Because the interfaces were relatively simple, it was possible to make many implementation changes with few interface changes. Shade uses two primary interfaces. One is constraintoriented (“what”) rather than programatic (“how”). Constraint-oriented interfaces are more tied to the ultimate problem being solved, and therefore more stable. The programatic interface is a callback to a user extension. That interface is also simple and relatively robust against changes in the Shade core. Despite requests from a few users, Shade does not provide an interface to augment the processor model by calling user functions to update the machine state. This omission is clearly a disadvantage from the perspective of some users, but relatively few users requested it and avoiding it keeps other things simple. There have otherwise been few requests about how to use the interface and few requests for more features. Shade does not read or write an ASCII form. Shade is not built on any infrastructures beyond the host operating system, but that is in part because Shade was built when there were few infrastructure packages for machine code parsing, machine code generation, construction of dynamic translators, or architecture descriptions. Such infrastructure should help with construction, porting, and improving reliabilty of Shade-like tools.

Improving Your Infrastructure: The implementation of Shade was regularly examined and improved on an ongoing basis. Also, the lessons of bugs past helped keep the implementation relatively reliable and the interface relatively clean. Many requests and complaints from users led to improvements in Shade, though some requests were explicitly refused. Shade was available only in binary form during much of its development, so it was difficult for users to contribute. Enthusiastic users within Sun often demanded the newest version, making it difficult to pursue perfection. There was also a conscious decision to pursue reliability and usability over peak performance. Shade was preceded by several tools which count towards “throw one away”. In addition, several significant design changes made it necessary to throw away and rewrite portions of Shade.

3.2 QuickThreads

QuickThreads is a toolkit for constructing user-level threads packages. It was built as a response to portability and performance problems with a large number of earlier one-off threads packages, which limited the tools that used them. QuickThreads was also built as an attempt to combine good features of several academic reasearch threads packages, without suffering their disadvantages. The design and use of QuickThreads is described elsewhere [Kep93b].

Non-Technical Costs: QuickThreads was developed under academic funding; the lack of ongoing support has hampered porting to new machines. Probably the most frequent communication from users is a request for a PowerPC port; due to no funding, it has been years in coming. QuickThreads is currently being updated for transfer to the Free Software Foundation, which will provide better support. There is no QuickThreads champion. Initial advertising was limited to a few USENET posts, largely in response to questions from people about portable threads packages. QuickThreads was subsequently picked up by others and is now in several canonical lists of threads packages.

Background Reasearch: The initial targets were a few people running a few specific tools, and an academic community that cared more about the pedagogical result than implementation or compatability. QuickThreads is distributed in source form and was written as much as possible in C for compatability with the preexisting tools. Since QuickThreads was written in response to perceived problems; the included documentation has comparisons to other threads packages.

Legal Restrictions: There are no legal restrictions on QuickThreads. In recent discussions about assigning it to the Free Software Foundation, we agreed to continue unrestriced distribution because it is already used in commercial (non-free) products. There has been little user-contributed code though a PowerPC port has been contributed with the understanding it would be contributed to the Free Software Foundation.

Encouraging Wide-Spread Use: QuickThreads is free and comes with complete source code for several demo programs. QuickThreads is distributed only in source form, which requires extra configuration labor by first-time users. Configuration is relatively straightforward: select between specified machine targets and platform types, then compile, which also builds the demo programs. Multiple distribution formats might be desirable, but lack of time and build machines has prevented that. QuickThreads is designed to provide machine-dependent code so portability is necessarily a problem. Varying assembler syntax among x86 platforms, and varying calling conventions for PowerPC convensions have been ongoing minor portability problems. QuickThreads comes with benchmarks which double as first-try correctness tests. They have uncovered a few problems; few bugs have been reported that got past the above tests. Examples are written in C. There is currently no user community; gifting it to the Free Software Foundation should solve that problem.

Writing Ancilary Text: QuickThreads comes with a single document which describes the implementation and the user interface. Two documents would be better. A separate porting guide is provided with the distribution. The distribution includes examples.

Bugs: QuickThreads is relatively simple and there have been only a few bugs reported. As noted above, QuickThreads comes with a few simple tests. The interface is a source of some user confusion, but it is not clear there is a better alternative. Probably the best remedy would be better documentation that included a step-by-step example. Although the interface is conceptually somewhat complicated, it is small and relatively robust against user errors. QuickThreads is small and has a simple implementation. As a result, there have been few bugs, and community debugging has largely not been an issue. There were no Alpha releases, but the package’s simplicity has meant that was less of an issue.

Interfaces: The QuickThreads interface design was a large part of the overall design. One of the biggest interface mistakes may have been supporting C variadic (variable-length) argument lists. Although the support was a useful and interesting part of the development and a good goal for research exploration, it has substantially complicated the implementation, discouraged others from building PowerPC ports, and few users have cited it as an benefit. Generality has been achieved by stripping wrappers and providing only the core; to some degree, generality has come at the expense of exposing a more complex interface model. There is no support for reading and writing ASCII: it is not clear what would be printed or read by such support. Extensibility was a key design goal of QuickThreads; it differs in that respect from essentially all other threads packages, and it uses a fundamentally different extension model [Kep93a] than the one other known extensible package. In particular, QuickThreads is not a complete threads package, it requires user extension for even minimal funcationality. QuickThreads is sufficiently low-level it is not clear what existing infrastructure it would be built on; a machine-readable description of the calling conventions of various processors would be tremendously useful, but none seems to be available.

Improving Your Infrastructure: The implementation is ugly, making it more difficult for others to debug or port QuickThreads. However, it remains unimproved due to overall lack of support. There have been only two releases, the second simply adding a new target to what was avaialble in the first release. Thus, there has been little evolution. User comments have been rare. There is no mechanism to contribute, beyond simply sending e-mail. The built-in tests were useful in a third party developing a PowerPC port. Gifting QuickThreads to the Free Software Foundation will improve evolution and contribution, and may lead to more user comments. QuickThreads is itself the result of throwing away several earlier threads packages.

4 Summary

Good infrastructure has many benefits but is also suprisingly difficult to get right. This paper presents various hints to help developers to build a good system, make it available and well know, reduce the barrier to entry for potential users, and to help users be satisifed with the infrastructure.

5 Acknowledgments

The beginnings of this note grew out of a panel session at the joint University of Washington/Microsoft Summer Software Institute. The panel reviewed the infrastructure components LCC, Machine SUIF, Shade, SimpleScalar and QuickThreads. We felt that these were examples of successful infrastructure, and we were also familiar with them, some people as creators or maintainers of the infrastructure and some as users of the infrastructure. The panel members were Todd Austin, Chris Fraser, Jonathan Hardwick, Robert Henry, David Kaeli, Pardo, Mike Smith, and Andreas Zeller.

References

[CK94] Robert F. Cmelik, and David Keppel, “Shade: A Fast Instruction-Set Simulator for Execution Profiling,” Proceedings of the 1994 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems May 1994, pp. 128-137.

[Kep93a] David Keppel, “Managing Abstraction-Induced Complexity”, University of Washington Technical Report #93-06-02, 1993.

[Lam84] Butler W. Lampson. “Hints for Computer System Design,” ACM Operating Systems Review, 15(5), October 1983, pp. 33-48. Reprinted in IEEE Software, 1(1) January 1984, pp. 11-28. See also http://research.microsoft.com/˜ lampson/33-Hints/WebPage.html as of 2003/03/26.

[FH95] Christopher Fraser and David R. Hanson. “A Retargetable C Compiler: Design and Implementation,” Addison-Wesley Publishing, Boston, MA, 1995.

[MachineSUIF] See http://www.eecs.harvard.edu/machsuif as of 2003/03/26.

[Kep93b] “Tools and Techniques for Building Fast Portable Threads Packages”, University of Washington Technical Report #93-05-06, May 1993.

[SPEC] See http://www.spec.org as of 2003/03/26.

[SimpleScalar] See http://www.SimpleScalar.com as of 2003/03/26.