Within each section the papers are in reverse chronological order, with the most recent at the top. Document sizes quoted should be treated as a rough guide, rather than definitive.
- This symbol flags both new papers and old papers recently made available.
- This symbol flags the papers I consider the most important.
Please send comments and feedback to
The documents contained in these directories are intended by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their work here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
Unless otherwise noted, the materials listed below are Copyright © <year-of-publication> Hewlett-Packard Development Company, L.P.
In the case of ACM copyrighted documents, the materials below represent the "author's version" of the works; the definitive versions may be obtained from the ACM Digital Library.
Please contact me if you would like to use any of the documents listed here.
Omega: flexible, scalable schedulers for large compute clusters. Malte Schwarzkopf, Andy Konwinski, and Michael Abd-el-Malek and John Wilkes. EuroSys'2013, April 2013 (Prague, Czech Republic). © ACM 2013. [Winner of best student paper at EuroSys.]
CPI2: CPU performance isolation for shared compute clusters. Xiao Zhang, Eric Tune, Robert Hagmann, Rohit Jnagal, Vrigo Gokhale, and John Wilkes (Google Inc.) EuroSys'2013, April 2013 (Prague, Czech Republic). © ACM 2013.
Related article: Google: 'We'll track EVERY task on EVERY data center server': Chip-level performance tracking in thousand-server Googly clusters. Jack Clark. The Register. 12 April 2013.
Obfuscatory obscanturism: making workload traces of commercially-sensitive systems safe to release [159KB]. [slides] Charles Reiss (University of California, Berkeley), John Wilkes, Joseph L. Hellerstein (Google). CloudMAN, May 2012 (Maui, HI). © IEEE 2012.
CloudScale: elastic resource scaling for multi-tenant cloud systems [653KB]. Zhiming Shen, Sethuraman Subbiah, and Xiaohui Gu (North Carolina State University) and John Wilkes. Symposium on Cloud Computing (SoCC), Oct. 2011 (Cascais, Portugal). [A follow-on to the PRESS paper (below).]
PRESS: PRedictive Elastic ReSource Scaling for cloud systems [310KB]. Zhenhuan Gong, Xiaohui Gu, and John Wilkes. 6th IEEE/IFIP International Conference on Network and Service Management (CNSM 2010), Oct. 2010 (Niagara Falls, Canada). © IEEE 2010. Best paper award.
Utility functions, prices, and negotiation [0.2MB]. John Wilkes. HP Laboratories Technical report HPL-2008-81, 6 July 2008.
Now published as a chapter of Market-oriented grid and utility computing, Rajkumar Buyya and Kris Bubendorfer (editors), Wiley series on parallel and distributed computing. ISBN: 978-0-470-28768-2. John Wiley & Sons, Hobokem, NJ, October 2009 (pages 67-88). Available from Wiley or Amazon.com.
Service contracts and aggregate utility functions [240KB]. Alvin AuYoung, Laura Grit, Janet Wiener, and John Wilkes. 15th IEEE International Symposium on High Performance Distributed Computing (HPDC-15), pp. 119-131, June 2006 (Paris, France). DOI 10.1109/HPDC.2006.1652143. © IEEE 2006.
An earlier version was published as Traveling to Rome: a retrospective on the journey (paper 194KB and presentation 2.9MB). John Wilkes. The Rise and Rise of the Declarative Datacentre (R2D2), a joint Microsoft/HP research meeting, Cambridge, UK, 12-13 May 2008. Microsoft Research technical report MSR-TR-2008-61, pages 49-52.
John Wilkes speaks out: on what the DB community needs to know about storage, how the DB and storage communities can join forces and change the world, and more. John Wilkes interviewed by Marianne Winslett. ACM SIGMOD Record 34(3):100-110, September 2005. DOI 10.1145/1084805.1084829. © ACM 2005.
Lessons and challenges in automating data dependability [151KB]. Kimberly Keeton, Dirk Beyer, Jeff Chase, Arif Merchant, Cipriano Santos and John Wilkes. 11th ACM SIGOPS European Workshop, September 2004 (Leuven, Belgium). © ACM 2004. DOI 10.1145/1133572.1133591.
Designing for disasters [313KB]. Kimberly Keeton, Cipriano Santos, Dirk Beyer, Jeffrey Chase and John Wilkes. Conference on File and Storage Technologies (FAST'04) San Francisco, CA, March-April 2004. Published by USENIX, Berkeley, CA.
Automatic design of dependable data storage systems [66KB]. Kimberly Keeton and John Wilkes. Workshop on Algorithms and Architectures for Self-managing Systems (Self-manage'03), San Diego, CA, June 2003, pp. 7-12.
Data services – from data to containers [presentation, 2-up, PDF 7.4MB]. John Wilkes. Conference on File and Storage Technologies (FAST'03) San Francisco, CA, March-April 2003. Published by USENIX, Berkeley, CA.
Automating data dependability [50KB]. Kimberly Keeton and John Wilkes. 10th ACM-SIGOPS European Workshop, September 2002 (Saint-Emilion, France), pages 93-100. DOI 10.1145/1133373.1133391. © ACM 2002.
Selecting RAID levels for disk arrays [733KB]. Eric Anderson, Ram Swaminathan, Alistair Veitch, Guillermo A. Alvarez and John Wilkes. Conference on File and Storage Technology (FAST'02), pages 189-201, 28-30 January 2002 (Monterey, CA). Published by USENIX, Berkeley, CA.
Aqueduct: online data migration with performance guarantees [733KB]. Eric Anderson, Ram Swaminathan, Alistair Veitch, Guillermo A. Alvarez and John Wilkes. Conference on File and Storage Technology (FAST'02) pages 219-230, 28-30 January 2002 (Monterey, CA). Published by USENIX, Berkeley, CA.
Appia: automatic storage area network design [505KB]. Julie Ward, Michael O'Sullivan, Troy Shahoumian, and John Wilkes. Conference on File and Storage Technology (FAST'02), pp. 203-217, 28-30 January 2002, Monterey, CA. Published by USENIX, Berkeley, CA.
Appia and the HP SAN Designer: automatic storage area network fabric design [252KB]. Julie Ward, Michael O'Sullivan, Troy Shahoumian, John Wilkes, Ren Wu, and Dirk Beyer. HP Technical Conference 2003, April 2003 (Keystone, CO). — An internal HP technical conference.
An experimental study of data migration algorithms [210KB]. Jared Saia, Eric Anderson, Joe Hall, Jason Hartline, Michael Hobbes, Anna Karlin, Ram Swaminathan, and John Wilkes. 5th Workshop on Algorithm Engineering (WAE 2001), 28-30 August 2001 (BRICS, University of Aarhus, Denmark). Published as Lecture Notes in Computer Science 2141, pp. 145-158, August 2001, Springer-Verlag. DOI 10.1007/3-540-44688-5. © 2001 Springer-Verlag.
Minerva: an automated resource provisioning tool for large-scale storage systems [232KB]. Guillermo A. Alvarez, Elizabeth Borowsky, Susie Go, Theodore H. Romer, Ralph Becker-Szendy, Richard Golding, Arif Merchant, Mirjana Spasojevic, Alistair Veitch, and John Wilkes. ACM Transactions on Computer Systems 19(4):483-518, Nov. 2001. DOI 10.1145/502912.502915. © 2001 ACM.
Traveling to Rome: QoS specifications for automated storage system management [166KB]. John Wilkes. Proc. Intl. Workshop on Quality of Service (IWQoS'2001) pages 75-91 (6-8 June 2001, Karlsruhe, Germany). Lecure Notes in Computer Science 2001. DOI 10.1007/3-540-45512-4_7. © 2001 Springer-Verlag.
On algorithms for efficient data migration [226KB]. Joseph Hall, Jason Hartline, Anna R. Karlin, Jared Saia, and John Wilkes. 12th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'01), 7-9 January 2001, Washington, DC.
Storage Systems Management [1.4MB]. Guillermo Alvarez, Kim Keeton, Arif Merchant, Erik Riedel, John Wilkes. Tutorial presented at International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2000), Santa Clara, CA, June 2000.
Capacity planning with phased workloads [410KB]. Elizabeth Borowsky, Richard Golding, Patricia Jacobson, Arif Merchant, Louis Schreier, Mirjana Spasojevic and John Wilkes. 1st International Workshop on Software and Performance (WOSP'98), Santa Fe, NM, October 1998. DOI 10.1145/287318.287363. © 1998 ACM. Presentation [338KB].
Using attribute-managed storage to achieve QoS [37KB]. E. Borowsky, R. Golding, A. Merchant, L. Schreier, E.Shriver, M.Spasojevic, and J. Wilkes. 5th Intl. Workshop on Quality of Service (IWQoS'97), June 1997 (Columbia Univ., New York, NY), pp. 199-202. © IEEE 1997.
Eliminating storage headaches through self-management [19KB]. Elizabeth Borowsky, Richard Golding, Arif Merchant, Elizabeth Shriver, Mirjana Spasojevic, and John Wilkes. Abstract for work-in-progress talk. 1996 Symposium on Operating Systems Design and Implementation (OSDI'96), Seattle, WA, 28-31 October 1996.
Automatic design of storage systems to meet availability requirements [95KB]. Khalil Amiri and John Wilkes. HP Laboratories technical report HPL-SSP-96-17, August 1996. Released externally in June 2008.
Attribute-managed storage [28KB]. Richard Golding, Elizabeth Shriver, Tim Sullivan, and John Wilkes. A position paper for the Workshop on Modeling and Specification of I/O, 26 Oct. 1995 (San Antonio, TX).
Stratified B-trees and versioning dictionaries [267KB]. Andy Twigg, Andrew Byde, Grzegorz Miło, Tim Moreton, John Wilkes, and Tom Wilkie (Acunu). HotStorage'11 (June 2011, Portland, OR). Published by USENIX, Berkeley, CA.
Storage, data, and information systems [Amazon.com] John Wilkes, Christopher Hoover, Beth Keer, Pankaj Mehra, and Alistair Veitch. HP Laboratories, Palo Alto, CA. ISBN-10: 1-4243-1731-2. 95 pages. 5th edition, January 2008. Available from Amazon.com.
Storage, data, and information: a brief overview. John Wilkes, Beth Keer, Christopher Hoover, Alistair Veitch, and Pankaj Mehra. Invited talk at National ICT Australia, Melbourne University, December 2006.
Hibernator: helping disk arrays sleep through the winter [640KB] Q. Zhu, Z. Chen, L. Tan, Y. Zhou, K. Keeton and J. Wilkes, Symposium on Operating Systems Principles (SOSP'05), pp. 177-190, October 2005. © ACM 2005.
Seneca: remote mirroring done write [219KB]. Minwen Ji, Alistair Veitch, and John Wilkes. USENIX Technical Technical Conference (USENIX'03) pp. 253-268 (June 2003, San Antonio, TX). Published by USENIX, Berkeley, CA. PowerPoint slides 441KB.
My cache or yours? Making storage more exclusive [251KB]. Theodore M. Wong and John Wilkes. USENIX Annual Technical Conference (USENIX 2002), pp 161-175, June 2002 (Monterey, CA). Presentation: 2up slides [161KB], with notes [204KB].
Towards global storage management and data placement [13KB]. 1-page position summary from 8th Workshop on Hot Topics in Operating Systems (HotOS VIII), 20-23 May 2001 (Schloss Elmau, Germany), © 2001 IEEE.
Full version: Towards global storage management and data placement [63KB]. Alistair Veitch, Erik Riedel, Simon Towers, and John Wilkes. HP Laboratories SSP technical report, HPL-SSP-2001-1, March 2001.
Persistent storage for distributed applications. Richard Golding and John Wilkes. 8th ACM SIGOPS European Workshop, 7-10 September 1998, pages 53-57 (Sintra, Portugal). DOI 10.1145/319195.319204. © ACM 1998.
An analytic behavior model for disk drives with readahead caches and request reordering [283KB]. Elizabeth Shriver, Arif Merchant and John Wilkes. SIGMETRICS'98, Madison, WI, June 1998. © ACM 1998.
Storage system function list. John Wilkes and Doug Voigt. Revision 5, 17 December 1997. — A list of the functions that might need to be distributed across a storage system architecture. No longer available.
On-line extraction of SCSI disk drive parameters [318KB]. Bruce L. Worthington, Gregory R. Ganger, Yale N. Patt and John Wilkes. HP Laboratories technical report HPL-97-02 (January 1997), and University of Michigan Technical report CSE-TR-323-96 (December 1996).
Strategic directions in storage I/O issues in large-scale computing. Garth A. Gibson, Jeffrey Scott Vitter, and John Wilkes. ACM Computing Surveys 28(4):779-793, December 1996. DOI 10.1145/242223.242300
- Self-Managing Network-Attached Storage. Garth Gibson and John Wilkes. ACM Computing Surveys 28(4es), December 1996 (updated Feb. 1997). DOI 10.1145/242224.242491. © ACM 1996. — Position statement prepared for survey paper listed above.
The HP AutoRAID hierarchical storage system [343KB]. John Wilkes, Richard Golding, Carl Staelin, and Tim Sullivan. ACM Transactions on Computer Systems 14 (1):108-136, February 1996. © ACM 1996.
- An earlier version [113KB] was presented at the 15th ACM Symposium on Operating System Principles (SOSP'95), Copper Mountain Resort, Colorado, 3-6 December 1995. Published as Operating Systems Review 29(5):96-108, © ACM 1995.
- Pantheon is the the event/trace-driven simulation system that we developed for the AutoRAID project. The software is available to qualified researchers from HP. Here are two papers about it.
- The Pantheon storage-system simulator [107KB]. John Wilkes. HP Laboratories technical report HPL-SSP-95-14, revision 1, May 1996.
- Oculus: a visual user interface for the Pantheon storage system simulator [334KB]. Uwe Aicheler. HP Laboratories technical report HPL-SSP-96-1, January 1996.
AFRAID – A Frequently Redundant Array of Independent Disks [111KB]. Stefan Savage and John Wilkes. 1996 USENIX Technical Conference, 22-26 January 1996, pages 27-39 (San Diego, CA). — "Best paper with a student author".
Frequently-redundant array of independent disks [1.2MB]. John Wilkes and Stefan Savage. US patent 5,720,025, granted 17 Feb. 1998, filed 18 Jan. 1996. — The AFRAID patent.
Techniques for file system simulation. Chandramohan A. Thekkath, John Wilkes and Edward D. Lazowska. Software—Practice and Experience 24(11):981-999, November 1994. © 1994 John Wiley and Sons Ltd. DOI 10.1002/spe.4380241102. — Available online from Wiley InterScience.
An introduction to disk drive modelling [134KB]. Chris Ruemmler and John Wilkes. Published in IEEE Computer 27(3):17-28, March 1994. © 1994 IEEE.
- An earlier version appeared as Modelling disks. Chris Ruemmler and John Wilkes. HP Laboratories technical report HPL-93-68rev1, 1993.
- David Kotz, Song Bac and Sriram Radhakrishnan at Dartmouth University later developed a simulation model of an HP97560 disk drive based on our description in this paper.
The TickerTAIP parallel RAID architecture [2.04MB]. Pei Cao, Swee Boon Lim, Shivakumar Venkataraman, John Wilkes. ACM Transactions on Computer Systems 12(3):236-267, August 1994. © ACM 1994.
UNIX disk access patterns [319KB]. Chris Ruemmler and John Wilkes. Winter'93 USENIX Conference, pages 405-420, January 1993.
- Also published as HP Laboratories Technical Report HPL-92-152 (December 1992).
- Another analysis of the same data can be found in: A trace-driven analysis of disk working set sizes [154KB]. Chris Ruemmler and John Wilkes. HP Laboratories Technical Report HPL-OSR-93-23, April 1993.
- The I/O traces we used are available to qualified researchers, although they are rather old now, and so shouldn't be used for new research.
Disk shuffling [269KB]. Chris Ruemmler and John Wilkes. HP Laboratories Technical Report HPL-91-156, October 1991.
Disk scheduling algorithms based on rotational position [516KB]. David M. Jacobson and John Wilkes. HP Laboratories technical report HPL-CSP-91-7rev1, February 1991, revised March 1991.
Improving the efficiency of UNIX file buffer caches [342KB]. Andrew Braunstein, Mark Riley, and John Wilkes. 12th ACM Symposium on Operating Systems Principles (SOSP'99), December 1989 (The Wigwam, Litchfield Park, AZ). Published as Operating Systems Review 23(5):71-82, December 1989. DOI 10.1145/74850.74858. © ACM 1989.
Specifying data availability in multi-device file systems [34KB]. John Wilkes and Raymie Stata. A position paper for the 4th ACM-SIGOPS European Workshop, 3-5 September 1990 (Bologna, Italy). Published as Operating Systems Review 25(1):56-59, January 1991. DOI 10.1145/504136.504139. © ACM 1991.
DataMesh was a project to investigate using a set of mesh-connected disk+compute nodes to build a scalable storage system.
DataMesh Architecture 1.0 [71KB]. Chia Chao, Robert English, David Jacobson, Bart Sears, Alexander Stepanov and John Wilkes. HP Laboratories technical report HPL-92-153, December 1992.
DataMesh, house-building, and distributed systems technology, John Wilkes. 5th ACM SIGOPS European workshop, 21-23 September 1992 (Mont Saint-Michel, France). DOI 10.1145/506378.506428. Published as Operating Systems Review 27(2):104-108, April 1993, DOI 10.1145/155848.155867.
Mime: a high performance parallel storage device with strong recovery guarantees [78KB]. Chia Chao, Robert English, David Jacobson, Alexander Stepanov and John Wilkes. HP Laboratories technical report HPL-CSP-92-9rev1, March 1992, revised November 1992.
DataMesh research project, phase 1. [36KB]. John Wilkes, with Chia Chao, Robert English, David Jacobson, Sai-Lai Lo, Chris Ruemmler, Bart Sears, Alex Stepanov, and Rebecca Wright. USENIX Workshop on File Systems, Ann Arbor, MI, pages 63-69, 21-22 May 1992.
DataMesh parallel storage servers. John Wilkes, Chia Chao, Robert English, David Jacobson, Bart Sears, Carl Staelin andAlex Stepanov. Operating Systems Review 26(2):11, April 1992. DOI
- DataMesh—scope and objectives [12KB]. John Wilkes. HP Laboratories technical report HPL-DSD-89-37rev1, 19 July 1989.
- DataMesh—scope and objectives: a commentary [84KB]. John Wilkes. HP Laboratories technical report HPL-DSD-89-44, 19 July 1989.
Towards a holistic approach to fault management: wheels within a wheel [460KB]. Moises Goldszmidt, Miroslaw Malek, Simin Nadjm-Tehrani,
Priya Narasimhan, Felix Salfner, Paul A. S. Ward, and John Wilkes.
This chapter appears in Dependability and Computer Engineering: Concepts for Software-Intensive Systems edited by Luigia Petre, Kaisa Sere, and Elena Troubitsyna (Åbo Akademi University, Finland). © 2011, IGI Global, www.igi-global.com. Posted by permission of the publisher. It was an outcome of a workshop on Self-healing and self-adaptive systems, Schloss Dagstuhl, May 2009.
The semiotics of umbrellas [163KB].
HP Laboratories, Palo Alto, CA, February 2003. Written for the collection Computer systems: papers for Roger Needham to mark 50 years in Cambridge and 5 years at Microsoft Research, edited by Andrew Herbert and Karen Spärck Jones, Cambridge, February 2003, Microsoft, pp. 257-260.
Back to the future: dependable computing = dependable services. Jeffrey Chase, Amin Vahdat, and John Wilkes. 10th ACM SIGOPS European workshop, pages 170-173, September 2002 (Saint-Emilion, France). DOI 10.1145/1133373.1133406. © 2002 ACM.
eOS – the dawn of the resource economy [97KB]. John Wilkes, Patrick Goldsack, G. (John) Janakiraman, Lance Russell, Sharad Singhal, and Andrew Thomas. This short version appeared as a position summary in the proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS VIII), 20-23 May 2001 (Schloss Elmau, Germany). DOI 10.1109/HOTOS.2001.990104. © 2001 IEEE.
Full version: eOS – the dawn of the resource economy [86KB]. HP Laboratories technical report, 21 May 2001.
Proceedings of the 17th ACM Symposium on Operating Systems Principles (SOSP'99). David Kotz (general chair) and John Wilkes (program chair), 12-15 December 1999 (Kiawah Island Resort, Charleston, South Carolina). Published as Operating Systems Review 33(5), December 1999. © 1999 ACM.
Is work hell? Life in industrial research [247KB]. John Wilkes. HP Laboratories technical report HPL-SSP-97-1, January 1997. — Slides and reader notes for the CMU "emigration course", designed to help graduating students look for a job.
"Tcl cures 98.3% of all known simulation configuration problems" claims astonished researcher! [26KB]. Richard Golding, Carl Staelin, Tim Sullivan, John Wilkes. Presented at the Tcl Workshop, New Orleans, May 1994.
- Describes the system we use in our Pantheon simulator to link Tcl and C++ code. This allows us to construct a wide variety of simulations from a kit of parts at runtime using an interpreted language, and yet still have the simulation execution run at full speed.
- Source code [73KB] for the scheme described in this paper is available from HP Labs.
The refdbms distributed bibliographic database system [128KB]. Richard Golding, Darrell Long and John Wilkes. Proceedings of Winter'94 USENIX Technical Conference, pages 47-62 (January 1994). © 1994 Usenix Association.
Brevix design 1.01 [315KB]. Martin Fouts, Tim Connors, Steve Hoyle, Bart Sears, Tim Sullivan, and John Wilkes. HP Laboratories technical report HPL-OSR-93-22, 1 April 1993. 80 pages.
A comparison of Protection Lookaside Buffers and the PA-RISC protection architecture [45KB]. John Wilkes and Bart Sears. HP Laboratories technical report HPL-92-55, March 1992.
CSP project startup documents [12KB]. John Wilkes. HP Laboratories technical report HPL-CSP-90-42, October 1990. 1 page. — A set of guidelines for helping to crystallize the purpose, goals and rationale of a new project or research activity.
Position statement for ACM SIGOPS European Workshop 1988: autonomy versus interdependence. John Wilkes. 1988 ACM SIGOPS European Workshop, 18-21 Sept. 1988 (Cambridge, UK). DOI 10.1145/504092.504133. © ACM 1988.
Do you know your IQ? A research agenda for Information Quality in systems [196KB]. Kimberly Keeton, Pankaj Mehra, John Wilkes. HotMETRICS'09(Seattle, WA), 19 June 2009. © The authors 2009.
Idleness is not sloth [62KB] (presentation [124KB]). Richard Golding, Peter Bosch, Carl Staelin, Tim Sullivan, and John Wilkes. Winter'95 USENIX Technical Conference, New Orleans, LA, 16-19 January 1995, pages 201-222. © 1995 Usenix Association.
Predictive power conservation [13KB]. John Wilkes. HP Laboratories technical report HPL-CSP-92-5, 14th Febuary 1992. 1 page.
An implementation of the Hamlyn sender-managed interface architecture [230KB], presentation [5797K]. Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich and John Wilkes. 2nd Symposium on Operating Systems Design and Implementation (OSDI'96), (Seattle, WA), 28-31 October 1996, pages 245-259. DOI 10.1145/238721.238784.
Inter-processor communication system in which messages are stored at locations specified by the sender [24KB] A. J. Wilkes. US patent number 5,448,698. Granted 5th Sept. 1995. — The first Hamlyn patent.
Hamlyn: a high-performance network interface with sender-based memory management [78KB]. Greg Buzzard, David Jacobson, Scott Marovich and John Wilkes. Hot Interconnects III Symposium, Stanford University, Palo Alto, CA, 10-12 August 1995.
Hamlyn—an interface for sender-based communications [103KB]. John Wilkes. HP Laboratories technical report HPL-OSR-92-13, November 1992. — The original Hamlyn paper.
The Rainbow workstation [no PDF]. A. J. Wilkes, D. W. Singer, J. J. Gibbons, T. R. King, P. Robinson, and N. E. Wiseman. Computer Journal 27(2):112-120, May 1984. DOI: 10.1093/comjnl/27.2.112. © 1982 British Computer Society.