Summer 2024
D10.3 Open Source Project DAPHNE Reference Implementation

In this deliverable, we describe the release of a DAPHNE open source reference
implementation. This encompasses the strategy behind it, the methodology we apply to realize this endeavor and the actual content of a released artifact. One core concept in DAPHNE’s methodology is being open and inclusive in our development process. Consequently, it appeared to us that starting the practice of releasing DAPHNE into open source on a continuous basis early on was more beneficial than waiting for it to mature behind the curtains.

Winter 2023/24


 D1.5 3rd Annual Project Report (WP1 /KNOW) [Report, Public]

In D1.5 DAPHNE project team describes the progress made from project month 25 to project month 36, respectively the work done in project year 3 (M25/Dec 2022 – M36/Nov 2023) regarding the strategic objectives and throughout all work packages.

D3.4 Compiler Design and Overview (WP3 / TUB) [Report, Public]

Previous deliverables already shared the overall and refined system architecture [D2.1, D2.2], the language design specification [D3.1], as well as the initial and extended compiler prototypes [D3.2, D3.3]. This document presents the DAPHNE compiler design and overview. The DAPHNE compiler is based on MLIR (multi-level intermediate representation) as a framework for domain-specific compilers to facilitate a cost-effective development of our domain-specific language, reuse of compiler infrastructure, and good extensibility. This document first provides the necessary background on MLIR.

D4.3 Improved DSL Runtime Prototype and Overview (WP4 / ICCS) [Report, Public]

In this deliverable, we describe the status of the DAPHNE runtime system and its prototype v2 implementation. Specifically, we overview the current system architecture and continue with the implementation and integration updates that took place within this reporting period. We can categorize progress made along the following axes: a) Advances in runtime I/O; b) updates in runtime communication and c) updates inside the execution engine itself. We provide access to the DAPHNE Runtime v2 prototype, publicly available in the DAPHNE development repository. We conclude this report by including examples on how to execute sample DSL algorithms with various parameters and provide a large set of benchmarking results of the current version over the VEGA supercomputer.

D5.3 Improved Prototype of Pipelines and Task Scheduling Mechanisms (WP5 / UNIBAS) [Demonstrator, Public]

This document describes the improved pipeline and task scheduling prototype implemented in the DAPHNE system. The information presented in this document is based on a snapshot of the DAPHNE system prototype that implements well-tested and verified scheduling strategies. The improved prototype of pipeline, task, and parameter server scheduling include usability improvements, the addition of automated scheduling to the DAPHNE scheduling framework (DAPHNEsched), the implementation of inter-node scheduling infrastructure, and parameter server updates.

D6.3 Prototype and Overview of Data Path Optimizations and Placement (WP6 / ITU) [Demonstrator, Public]

Report and prototype of used data path optimization techniques and automatic data placement in hybrid memory and storage configurations. This deliverable describes the second version of the demonstrator of the Delilah computational storage prototype. The report also describes data path optimizations and placement in the context of the DAPHNE storage subsystem, including hardware acceleration on the data path.

D7.3 Prototype and Overview Code Generation Framework (WP7 / TUD) [Demonstrator, Public]

As introduced in DAPHNE report D7.1 , the challenges for the integration of HW accelerators are (i) developing as well as generating operators – hereafter also called computation kernels or kernels for short – which can be efficiently executed on accelerators such as CPUs, GPUs or FPGAs, (ii) integrating these accelerator-specific operators in the whole DAPHNE compilation and runtime infrastructure in a seamless way, and (iii) selecting the bestfitting accelerator for efficient execution depending on the specific IDA pipeline and hardware environment [D7.1].

D8.2 Improved Pipelines all Use Case Studies (WP8 / KAI) [Report, Public]

The use case pipelines presented in the DAPHNE project are enhanced by utilizing the DAPHNE system infrastructure. Depending on the complexity of the initial pipeline description, the individual use cases either chose for implementing the pipeline in DSL or DaphneLib. The WP8 project partners contributed to the development of DAPHNE system infrastructure by providing constructive feedback in the form of GitHub issues as well as pull-requests to the source code repository.

Summer 2023

D3.3 Extended Compiler Prototype

Previous deliverables already shared the overall and refined system architecture [D2.1, D2.2] as well as the initial language and compiler designs [D3.1, D+22] and the initial compiler prototype [D3.2]. This document presents the extended DAPHNE compiler prototype, which is still based on MLIR (multi-level intermediate representation) as a library of compiler infrastructure to facilitate a cost-effective development of our domain-specific language, reuse of compiler infrastructure, and good extensibility.

D9.3 Initial Prototype of Benchmarking Toolkit

This deliverable describes the implementation of the Universal Machine Learning Analysis Utility (UMLAUT) prototype. We describe the implementation of our design decisions, the use cases for the system, as well as its performance implications on Integrated Data Analysis (IDA) pipelines.

Winter 2022/23

These deliverables have been submitted but are not yet accepted, they might be revised after the next review period.

D1.4 2nd Annual Project Report

In D1.4 DAPHNE project team describe the progress made until project month 24 and here particularly the work done in project year 2. This report presents an overview of the type and purpose of the document, its revision history, the strategic objectives of DAPHNE project and the work carried out in project year 2 to reach these objectives.

D4.2 DSL Runtime Prototype

In this deliverable we demonstrate the use of the DAPHNE runtime by sharing a snapshot of the DAPHNE prototype, and provide an example scenario of running simple DaphneDSL scripts on the local runtime as well as on the distributed runtime.

D5.2 Prototype of Pipelines and Task Scheduling Mechanisms

This document describes the use of the pipeline and task scheduling mechanisms currently supported in the DAPHNE system. As discussed in previous deliverable reports [D4.1, D5.1], the DAPHNE project team continuously refines, extends, and adds scheduling mechanisms in DAPHNE. Nevertheless, extensibility is at the heart of the DAPHNE system.

D6.2 Prototype and Overview of Managed Storage Tiers and Near-Data Processing

This deliverable describes the design of the Delilah prototype for offloading eBPF code on a computational storage platform. We detail its implementation on the Daisy computational storage platform and discuss its initial performance evaluation.

Click here for direct access to download the demo

D7.2 Prototype and Overview HW Accelerator Support and Performance Models

This document describes a snapshot of our DAPHNE prototype regarding HW acceleration as a follow-up of deliverable D7.1. In particular, we introduce three example scenarios as the main drivers for the current prototype and describe how to execute these examples with enhanced hardware support.

D9.2 Initial Benchmark Concept and Definition

In D9.2, HPI presents the initial concept and definition of the benchmark for the DAPHNE system.

Summer 2022

D2.2 Refined System Architecture

D2.2 is based on D2.1 Initial System Architecture (see below) and illustrated how this initial system architecture is retained. On top of that we have integrated earlier lessons learned and provide more details on several components.

D7.1 Design of integration HW accelerators

In D7.1, the Daphne project team reports on the planned overall design of integration HW accelerators as well as details on accelerated operations and primitives, as well as its compiler and runtime support.

Winter 2021/2022

D1.3 First Annual Project Report

This report summarizes the work carried out throughout the first year of the Daphne project, as well as the progress and initial results obtained so far. In detail, we share the overall work and progress towards the objectives, status of groups of work packages, and impact.
Additionally, this report also includes an update of the exploitation and dissemination plan, as well as the research data management plan.

D3.1 Language Design Specification

This report on the language design specification summarizes the design of DAPHNE’s domain-specific language (DSL) and related user-facing application programming interfaces (APIs).

D3.2 Compiler Prototype

This document shares a snapshot of this prototype, and describes an example scenario of running regression (and similarly classification, clustering, dimensionality reduction, and graph processing) algorithms on real datasets. The initial open source release of the DAPHNE prototype system is planned for end of March 2022.

D4.1 DSL Runtime Design

In this report the initial design of the Daphne runtime system and its prototype implementation are addressed. Moreover, the support for distribution primitives and the integration with existing frameworks are outlined.

D5.1 Scheduler Design for Pipelines and Tasks

This report describes the Daphne system based scheduling context and decisions as well as the preliminary design of the Daphne scheduler for pipelines and tasks.

D6.1 Computational Storage Capabilities

This report explains how compute and storage interact within storage devices and across a cluster, examining  the recent evolutions in storage architectures and programming abstractions.

Summer 2021

D2.1 Initial System Architecture

This report on the initial system architecture summarizes the preliminary design of the Daphne system infrastructure.

D8.1 Initial Pipeline Definition of all Use Cases

This report describes the five use case pipelines of earth observation, semiconductor manufacturing, material degradation and automotive vehicle development, whereas one is focussing on ejector geometry observation and the latter on virtual prototype development.