AI Surrogate Models LDRD

Running Legacy Code on Heterogeneous Hardware via Surrogate Models

Project Description

We propose to develop tools to automatically generate machine learning surrogate models from existing software so that they may utilize modern heterogeneous hardware. A hypothetical future high performance data facility would make extensive use of heterogeneous hardware such as GPUs and FPGAs, but many legacy codes will need to be heavily modified in order to take advantage of this hardware. Surrogate models replace a piece of code which is expensive to run with an approximate model; when the underlying model is a neural net, it runs efficiently on heterogeneous hardware. Thus, they are a promising technique for offloading computation to such hardware while minimizing the necessary changes to the original code. The tools developed during this project would make it substantially simpler to implement a surrogate model, enabling legacy code to access heterogeneous hardware, saving users' time and effort, and eliminating redundant work. Lowering these barriers should enable faster development iterations and make it easier to bring machine learning research code into production. This project includes a proof of principle of a new kind of code analysis tool which could be useful for an even broader variety of problems in high performance computing. Several of the milestones open up opportunities for future research into neural differential equations and automatic identification of functions to surrogate.

General Resources

• Source code lives here: https://github.com/nathanwbrei/phasm
• SRGS 2022
• Project planning (Trello page)

Meetings

Thursday 2-3pm, bi-weekly. Teams.

Meeting ID: 281 724 015 543

Passcode: cWidgE

Documents

Proposals

• LDRD proposal 2022
• LDRD proposal 2023

Presentations

Date	Event	Presenter	Slides
2022-06-27	SRGS 2022	Nathan Brei, David Lawrence	PDF

Publications

Date	Journal	Title
ccc	aaa	bbb

Notes

• The NVIDIA GPU profiling tool for the GPU Roofline analysis, Xinxin Mei
• Utilizing Advisor and Likwid for CPU Roofline analysis, Xinxin Mei

Actionables

• Surrogate toolkit: Implement array inputs and test on a 2D PDE such as diffusion. Nathan

• Vacuum tool: Automatically identify primitive inputs and outputs from the function signature. Nathan

• Tool for profiling the memory bounds on a GPU. David + Nathan

• Start thinking about a neural net model for a 2D PDE. Kishan + Nathan

• Improve the existing GlueX tracking models in preparation for Q3 and Q4 milestones Kishan

Useful Links

• Roofline Performance Model: https://crd.lbl.gov/divisions/amcr/computer-science-amcr/par/research/roofline/
• PINN: https://www.sciencedirect.com/science/article/pii/S0021999118307125