SMART APPLICATIONS (SmartApps) is a framework for application-centric computing. The overriding philosophy of SmartApps is "measure, compare and adapt if beneficial." The application will continually monitor its performance and the available resources to determine if, and by how much, performance can be improved by restructuring the problem. If the potential performance outweighs the projected overhead costs, SmartApps will perform an instance specific optimization. This optimization is more tractable than a global optimization between the application, OS and system.

Motivation

The approach in current distributed systems is to design and optimize the different components i.e. applications, compilers, operating systems (OS), hardware (HW) configurations in isolation without considering the needs of a running application or its instance-specific information. This leads to poor performance of many important applications. Dynamic changes to either the application or the system when the application is running makes static analysis and quality code generation difficult.

Operating Systems such as IBM's K42 can configure its services to meet the different characteristics of an application in every phase of its computation. Thus, customized performance can be achieved for every instance of the application. With progress in OS and HW with re-configuration possible, we need strategies and technologies to exploit them for improved application performance.

SmartApps Framework

The SmartApps framework includes performance monitoring and modeling components plus mechanisms for performing the actual restructuring at various levels including:

Algorithmic Adaptation
Run-Time Software Optimization (e.g. input sensitivity analysis, etc.)
Tuning re-configurable OS services (scheduling policy, page size, etc)
System configuration (e.g. selecting which, and how many, computational resources to use

SmartApps is being developed in the STAPL infrastructure. The Standard Template Adaptive Parallel Library is a framework for developing highly-optimizable, adaptable, and portable parallel and distributed applications. It consists of a relatively new and still evolving collection of generic parallel algorithms and distributed containers and a run-time system (RTS) through which the application and compiler interact with the OS and hardware.

(SMART) Applications

We will ensure the applicability and validity of the SmartApps approach by testing it on important computational sciences applications throughout the development process:

Computational Physics

Discrete-Ordinates Code which simulates subatomic particle transport being developed under the ASCI DOE program

Computational Biology

Protein Folding

Parallel Motion Planning
Seismic Ray Tracing

Development Platforms

SmartApps is being developed to run on current distributed, heterogeneous platforms, including the ASCI machines, large NUMA machines, and network of workstations that run under Linux and K42 operating systems. Future studies will be devoted to systems developed under the DARPA funded PERCS program. We will use the MAMBO simulator to experiment with re-configurable hardware platforms.

Publications and Presentations

An Algorithmic Approach to Communication Reduction in Parallel Graph Algorithms (Conference Best Paper Finalist), Harshvardhan, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Proc. IEEE Int.Conf. on Parallel Architectures and Compilation Techniques (PACT), San Francisco, CA, Oct 2015.
Proceedings(pdf, abstract)

Composing Algorithmic Skeletons to Express High-Performance Scientific Applications (Conference Best Paper), Mani Zandifar, Mustafa Abdujabbar, Alireza Majidi, David Keyes, Nancy M. Amato, Lawrence Rauchwerger, In Proc. ACM Int. Conf. Supercomputing (ICS), pp. 415--424, Newport Beach, CA, USA, Jun 2015.
Proceedings(pdf, abstract)

STAPL-RTS: An Application Driven Runtime System, Ioannis Papadopoulos, Nathan Thomas, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Proc. ACM Int. Conf. Supercomputing (ICS), pp. 425-434 , Newport Beach, CA, USA, Jun 2015.
Proceedings(pdf, ppt, abstract)

A Hybrid Approach To Processing Big Data Graphs on Memory-Restricted Systems, Harshvardhan, Brandon West, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Proc. Int. Par. and Dist. Proc. Symp. (IPDPS), pp. 799-808, Hyderabad, India, May 2015.
Proceedings(pdf, abstract)

A Hierarchical Approach to Reducing Communication in Parallel Graph Algorithms, Harshvardhan, Nancy M. Amato, Lawrence Rauchwerger, In Proc. ACM SIGPLAN Symp. Prin. Prac. Par. Prog. (PPOPP), pp. 285-286 (Poster), San Francisco, CA, USA, Jan 2015.
Proceedings(pdf, abstract)

Efficient, Reachability-based, Parallel Algorithms for Finding Strongly Connected Components, Daniel Tomkins, Timmie Smith, Nancy M. Amato, Lawrence Rauchwerger, Technical Report, TR15-002, Parasol Laboratory, Department of Computer Science, Texas A&M University, College Station, TX 77843-3112, Jan 2015.
Technical Report(pdf, abstract)

The STAPL Skeleton Framework, Mani Zandifar, Nathan Thomas, Nancy M. Amato, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), pp. 176--190, Hillsboro, OR, USA, Sep 2014.
Proceedings(pdf, abstract)

From Petascale to the Pocket: Adaptively Scaling Parallel Programs for Mobile SoCs, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Proc. IEEE Int.Conf. on Parallel Architectures and Compilation Techniques (PACT), SRC Poster, Aug 2014.
Proceedings(pdf)

KLA: A New Algorithmic Paradigm for Parallel Graph Computations (Conference Best Paper), Harshvardhan, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Proc. IEEE Int.Conf. on Parallel Architectures and Compilation Techniques (PACT), pp. 27-38, Edmonton, AB, Canada, Aug 2014.
Proceedings(pdf, abstract)

Processing Big Data Graphs on Memory-Restricted Systems, Harshvardhan, Nancy M. Amato, Lawrence Rauchwerger, In Proc. IEEE Int.Conf. on Parallel Architectures and Compilation Techniques (PACT), pp. 517-518 (SRC Poster), Edmonton, AB, Canada, Aug 2014.
Proceedings(pdf, abstract)

Using Load Balancing to Scalably Parallelize Sampling-Based Motion Planning Algorithms, Adam Fidel, Sam Ade Jacobs, Shishir Sharma, Nancy M. Amato, Lawrence Rauchwerger, In Proc. Int. Par. and Dist. Proc. Symp. (IPDPS), Phoenix, Arizona, USA, May 2014.
Proceedings(pdf, abstract)

A Scalable Framework for Parallelizing Sampling-Based Motion Planning Algorithms, Sam Ade Jacobs, Ph.D. Thesis, Department of Computer Science and Engineering, Texas A&M University, May 2014.
Ph.D. Thesis(pdf, abstract)

A Scalable Distributed RRT for Motion Planning, Sam Ade Jacobs, Nicholas Stradford, Cesar Rodriguez, Shawna Thomas, Nancy M. Amato, In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pp. 5088-5095, Karlsruhe, Germany, May 2013.
Proceedings(ps, pdf, abstract)

Load Balancing Techniques for Scalable Parallelization of Sampling-Based Motion Planning Algorithms, Adam Fidel, Sam Ade Jacobs, Shishir Sharma, Lawrence Rauchwerger, Nancy M. Amato, Technical Report, TR13-002 , Parasol Laboratory, Department of Computer Science, Texas A&M University, Mar 2013.
Technical Report(pdf, abstract)

The STAPL Parallel Graph Library, Harshvardhan, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Tokyo, Japan, Sep 2012.
Proceedings(pdf, abstract)

A Scalable Method for Parallelizing Sampling-Based Motion Planning Algorithms, Sam Ade Jacobs, Kasra Manavi, Juan Burgos, Jory Denny, Shawna Thomas, Nancy M. Amato, In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pp. 2529-2536, St. Paul, Minnesota, USA, May 2012.
Proceedings(ps, pdf, abstract)

From Days to Seconds: Scalable Parallel Algorithms for Motion Planning, Sam Ade Jacobs, Nancy M. Amato, In ACM Student Research Compet, Conf. on High Performance Computing Networking, Storage and Analysis Companion Proceedings, Seattle, Washington, USA, Nov 2011.
Proceedings(pdf, abstract)

The STAPL Parallel Container Framework, Gabriel Tanase, Antal Buss, Adam Fidel, Harshvardhan, Ioannis Papadopoulos, Olga Pearce, Timmie Smith, Nathan Thomas, Xiabing Xu, Nedhal Mourad, Jeremy Vu, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, In Proc. ACM SIGPLAN Symp. Prin. Prac. Par. Prog. (PPOPP), Feb 2011.
Proceedings(pdf, abstract)

The STAPL pView, Antal Buss, Adam Fidel, Harshvardhan, Timmie Smith, Gabriel Tanase, Nathan Thomas, Xiabing Xu, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Oct 2010. Also, Technical Report, TR10-001, Parasol Laboratory, Department of Computer Science, Texas A&M University, Jul 2010.
Proceedings(pdf, abstract) Technical Report(pdf, abstract)

STAPL: Standard Template Adaptive Parallel Library, Antal Buss, Harshvardhan, Ioannis Papadopoulos, Olga Tkachyshyn, Timmie Smith, Gabriel Tanase, Nathan Thomas, Xiabing Xu, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, In Haifa Experimental Systems Conference, Haifa, Israel, May 2010.
Proceedings(pdf, abstract)

The STAPL pList, Gabriel Tanase, Xiabing Xu, Antal Buss, Harshvardhan, Ioannis Papadopoulos, Olga Tkachyshyn, Timmie Smith, Nathan Thomas, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Newark, Delaware, Oct 2009.
Proceedings(pdf, abstract)

Two Memory Allocators that Use Hints to Improve Locality, Alin Jula, Lawrence Rauchwerger, In International Symposium on Memory Management, Trinity College Dublin, Dublin, Ireland, Jun 2009.
Proceedings(pdf, abstract)

Design for Interoperability in STAPL : pMatrices and Linear Algebra Algorithms, Antal Buss, Timmie Smith, Gabriel Tanase, Nathan Thomas, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Jul 2008.
Proceedings(pdf, abstract)

The STAPL pArray, Gabriel Tanase, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, In Proc. of Workshop MEDEA, pp. 81-88, Brasov, Romania, Sep 2007.
Proceedings(pdf, abstract)

Sensitivity Analysis for Automatic Parallelization on Multi-Cores, Silvius Rus, Maikel Pennings, Lawrence Rauchwerger, In Proc. ACM Int. Conf. Supercomputing (ICS), pp. 11, Jun 2007.
Proceedings(pdf, abstract)

Custom Memory Allocation for Free: Improving Data Locality with Container-Centric Memory Allocation, Alin Jula, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), New Orleans, Louisiana, Nov 2006.
Proceedings(pdf, abstract)

ARMI: A High Level Communication Library for STAPL, Nathan Thomas, Steven Saunders, Tim Smith, Gabriel Tanase, Lawrence Rauchwerger, Parallel Processing Letters, 16(2):261-280, Jun 2006.
Journal(ps, pdf, abstract)

SmartApps: Middleware for Adaptive Applications on Reconfigurable Platforms, Lawrence Rauchwerger, Nancy Amato, Operating System Reviews, 40(2):73-82, Apr 2006.
Journal(pdf, abstract)

Defero - a Semantic Driven Memory Allocator, Alin Jula, Lawrence Rauchwerger, Technical Report, TR06-001, Department of Computer Science and Engineering, Texas A&M University, Parasol Lab, Department of Computer Science, Texas A&M University, College Station, TX 77843-3112, Jan 2006.
Technical Report(abstract)

A Framework for Adaptive Algorithm Selection in STAPL, Nathan Thomas, Gabriel Tanase, Olga Tkachyshyn, Jack Perdue, Nancy M. Amato, Lawrence Rauchwerger, In Proc. ACM SIGPLAN Symp. Prin. Prac. Par. Prog. (PPOPP), pp. 277-288, Chicago, Illinois, Jun 2005.
Proceedings(ps, pdf, abstract)

An Adaptive Algorithm Selection Framework, Hao Yu, Dongmin Zhang, Lawrence Rauchwerger, In Proc. IEEE Int.Conf. on Parallel Architectures and Compilation Techniques (PACT), Antibes Juan-les-Pins, France, Sep 2004.
Proceedings(ps, pdf, abstract)

Run-time Optimizations of Adaptive Irregular Applications, Hao Yu, Ph.D. Thesis, Parasol Laboratory, Department of Computer Science, Texas A&M University, Aug 2004.
Ph.D. Thesis(ps, pdf, abstract)

An Adaptive Algorithm Selection Framework, Hao Yu, Dongmin Zhang, Francis Dang, Lawrence Rauchwerger, Technical Report, TR04-002, Parasol Laboratory, Department of Computer Science, Texas A&M University, Mar 2004.
Technical Report(ps, pdf, abstract)

SmartApps, An Application Centric Approach to High Performance Computing: Compiler-Assisted Software and Hardware Support for Reduction Operations, Francis Dang, Maria Jesus Garzaran, Milos Prvulovic, Ye Zhang, Alin Jula, Hao Yu, Nancy Amato, Lawrence Rauchwerger, Josep Torrellas, In Proc. Int. Par. and Dist. Proc. Symp. (IPDPS), pp. 172-181, Fort Lauderdale, Florida, Apr 2002.
Proceedings(ps, pdf, abstract)

Adaptive Parallel Sorting in the STAPL library, Steven Saunders, Nathan Thomas, Nancy Amato, Lawrence Rauchwerger, Technical Report, TR01-005, Department of Computer Science and Engineering, Texas A&M University, Nov 2001.
Technical Report(abstract)

Architectural Support for Parallel Reductions in Scalable Shared-Memory Multiprocessors, Maria Jesus Garzaran, Milos Prvulovic, Ye Zhang, Alin Jula, Hao Yu, Lawrence Rauchwerger, Josep Torrellas, In Proc. IEEE Int.Conf. on Parallel Architectures and Compilation Techniques (PACT), Barcelona, Spain, Sep 2001.
Proceedings(ps, pdf, abstract)

SmartApps: An Application Centric Approach to High Performance Computing, Lawrence Rauchwerger, Nancy Amato, Josep Torrellas, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Yorktown Heights, New York, USA, Aug 2000.
Proceedings(ps, pdf, abstract)

Run-time Parallelization Optimization Techniques, Hao Yu, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), San Diego, CA, Aug 1999.
Proceedings(ps, pdf, abstract)

Bounded Asynchrony and Nested Parallelism for Scalable Graph Processing, Adam Fidel, Nancy M. Amato, Lawrence Rauchwerger, In Proc. Supercomputing (SC), Doctoral Showcase Poster, Nov 2017.
Proceedings(pdf, abstract)

Fast Approximate Distance Queries in Unweighted Graphs using Bounded Asynchrony, Adam Fidel, Francisco Coral, Colton Riedel, Nancy M. Amato, Lawrence Rauchwerger, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Sep 2016.
Proceedings(pdf, abstract)

Design for Interoperability in STAPL: pMatrices and Linear Algebra Algorithms, Antal Buss, Timmie Smith, Gabriel Tanase, Nathan Thomas, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, Lecture Notes in Computer Science, 5335/2008:304-315, 2008.
Journal(pdf, abstract)

Associative Parallel Containers In STAPL, Gabriel Tanase, Chidambareswaran (Chids) Raman, Mauro Bianco, Nancy M. Amato, Lawrence Rauchwerger, Lecture Notes in Computer Science, 5234/2008:156-171, 2008. Also, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), Oct 2007.
Journal(pdf, abstract) Proceedings(pdf, abstract)

Design and Use of htalib - A Library for Hierarchically Tiled Arrays, Ganesh Bikshandy, Jia Guo, Christoph von Praun, Gabriel Tanase, Basilio Fraguela, Maria Jesus Garzaran, David Padua, Lawrence Rauchwerger, Lecture Notes in Computer Science, 4382/2007:17-32, 2008.
Journal(pdf, abstract)

Parallel Protein Folding with STAPL, Shawna Thomas, Gabriel Tanase, Lucia K. Dale, Jose M. Moreira, Lawrence Rauchwerger, Nancy M. Amato, Concurrency and Computation: Practice and Experience, 17(14):1643-1656, Dec 2005.
Journal(ps, pdf, abstract)

Parallel Protein Folding with STAPL, Shawna Thomas, Nancy M. Amato, In Proc. IEEE Int. Wkshp. on High Performance Computational Biology, Santa Fe, NM, Apr 2004.
Proceedings(ps, pdf, abstract)

ARMI: An Adaptive, Platform Independent Communication Library, Steven Saunders, Lawrence Rauchwerger, In Proc. ACM SIGPLAN Symp. Prin. Prac. Par. Prog. (PPOPP), pp. 12, San Diego, CA, Jun 2003.
Proceedings(ps, pdf, abstract)

STAPL: An Adaptive, Generic Parallel C++ Library, Ping An, Alin Jula, Silvius Rus, Steven Saunders, Tim Smith, Gabriel Tanase, Nathan Thomas, Nancy Amato, Lawrence Rauchwerger, Lecture Notes in Computer Science, 2624/2003:195-210, 2003. Also, In Wkshp. on Lang. and Comp. for Par. Comp. (LCPC), pp. 193-208, Cumberland Falls, Kentucky, Aug 2001.
Journal(pdf, abstract) Proceedings(ps, pdf, abstract)

Object Oriented Abstractions for Communication in Parallel Programs, Steven Saunders, Masters Thesis, Parasol Laboratory, Department of Computer Science, Texas A&M University, College Station, TX, May 2003.
Masters Thesis(ps, pdf, abstract)

A parallel communication infrastructure for STAPL, Steven Saunders, Lawrence Rauchwerger, In Wkshp. on Perf. Opt. for High-Level Lang. and Lib. (POHLL), New York, NY, Jun 2002.
Proceedings(ps, pdf, ppt, abstract)

STAPL: A Standard Template Adaptive Parallel C++ Library, Ping An, Alin Jula, Silvius Rus, Steven Saunders, Tim Smith, Gabriel Tanase, Nathan Thomas, Nancy Amato, Lawrence Rauchwerger, In Int. Wkshp on Adv. Compiler Technology for High Perf. and Embedded Processors, pp. 10, Bucharest, Romania, Jul 2001.
Proceedings(ps, pdf, abstract)

Standard Templates Adaptive Parallel Library (STAPL), Lawrence Rauchwerger, Francisco Arzu, K Ouchi, In Wkshp. on Lang. Comp. and Run-time Sys. for Scal. Comp. (LCR)., pp. 402-410, LNCS 1511. Pittsburgh, PA, May 1998.
Proceedings(ps, pdf, abstract)