Asset Details

MbrlCatalogueTitleDetail

Do you wish to reserve the book?

Scaling up ATLAS production system for the LHC Run 2 and beyond: project ProdSys2

by Vaniachine, A , Klimentov, A , Garcia, J , Golubkov, D , Borodin, M , Navarro , De, K , Maeno, T

in Applications programs / Big Data / Data base management systems / Data processing / Physics / Scaling up / Transformations / Workflow

2015

Yes Please

Hey, we have placed the reservation for you!

By the way, why not check out events that you can attend while you pick your title.

Oops! Something went wrong.

Looks like we were not able to place the reservation. Kindly try again later.

Are you sure you want to remove the book from the shelf?

Scaling up ATLAS production system for the LHC Run 2 and beyond: project ProdSys2

by Vaniachine, A , Klimentov, A , Garcia, J , Golubkov, D , Borodin, M , Navarro , De, K , Maeno, T

in Applications programs / Big Data / Data base management systems / Data processing / Physics / Scaling up / Transformations / Workflow

2015

Confirm

Do you wish to request the book?

Scaling up ATLAS production system for the LHC Run 2 and beyond: project ProdSys2

by Vaniachine, A , Klimentov, A , Garcia, J , Golubkov, D , Borodin, M , Navarro , De, K , Maeno, T

in Applications programs / Big Data / Data base management systems / Data processing / Physics / Scaling up / Transformations / Workflow

2015

Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy

How would you like to get it?

Submit

We have requested the book for you!

Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.

Oops! Something went wrong.

Looks like we were not able to place your request. Kindly try again later.

Journal Article

Scaling up ATLAS production system for the LHC Run 2 and beyond: project ProdSys2

Vaniachine, A,

Klimentov, A,

Garcia, J,

Golubkov, D,

Borodin, M,

Navarro,

De, K,

Maeno, T

2015

Overview

The Big Data processing needs of the ATLAS experiment grow continuously, as more data and more use cases emerge. For Big Data processing the ATLAS experiment adopted the data transformation approach, where software applications transform the input data into outputs. In the ATLAS production system, each data transformation is represented by a task, a collection of many jobs, submitted by the ATLAS workload management system (PanDA) and executed on the Grid. Our experience shows that the rate of task submission grows exponentially over the years. To scale up the ATLAS production system for new challenges, we started the ProdSys2 project. PanDA has been upgraded with the Job Execution and Definition Interface (JEDI). Patterns in ATLAS data transformation workflows composed of many tasks provided a scalable production system framework for template definitions of the many-tasks workflows. These workflows are being implemented in the Database Engine for Tasks (DEfT) that generates individual tasks for processing by JEDI. We report on the ATLAS experience with many-task workflow patterns in preparation for the LHC Run 2.

Share this book

Add to My Shelf

Publisher

IOP Publishing

Subject

Applications programs

/ Big Data

/ Data base management systems

/ Data processing

/ Physics

/ Scaling up

/ Transformations