Structural Funds in Poland Polish NGI and Cyfronet case study

Transkrypt

1 1 Domain-oriented services and resources of Polish Infrastructure for Supporting Computational Science in the European Research Space Structural Funds in Poland Polish NGI and Cyfronet case study Jacek Kitowski ACK Cyfronet AGH, Krakow, Poland PL-Grid Consortium EC Workshop Structural Funds, Brussels,

2 Outline 2 Motivation and Background Consortium Family of PL-Grid Projects Domain-specific solutions and services Overview Use-cases Expertise Conclusions

3 Motivation and Background 3 Computational Science problems to be addressed: (initiative from scientific community) Algoritms, environments and deployment 4 th paradigm, Big Data, Data Farming Domain-Specific solutions and environments PL-Grid Consortium (2007) 5 Polish Computer Centres Continuous development based on: Projects funded by the European Regional Development Fund as part of the Innovative Economy Program Polish Optical Internet (PIONIER) (2x10+ Gb/s) Close international collaboration (EGEE, EGI InSPIRE, PRACE, ESFRI projects, e.g., CTA, EPOS, ERIC cases..) RoadMaps for e-infrastructure developments Previous projects (5FP, 6FP, 7FP, EDA ) Integration with international platforms

4 Projects coordinated by Cyfronet 4 PL-Grid Project ( ) (run by PL-Grid Consortium) Budget: total 21 M, from EU 17M Number of people involved: ca. 80 (total, from different Polish Centres) Outcome: Common base infrastructure National Grid Infrastructure (NGI_PL) Resources: 230 Tflops, 3.6 PB PLGrid PLUS Project ( ) (run by PL-Grid Consortium) Budget: total ca.18 M, from EU: ca.15 M Number of people involved: ca. 120 IT scientists Computational scientists Field experts Expected outcome: Focus on users Domain specific solutions: 13 domains (Specific computing environments) QoS by SLM Extension of resources and services by: 500 Tflops, 4.4 PB Keeping diversity for users Clusters (thin and thick nodes, GPU), SMP, vsmp Clouds (OpenStack, Open Nebula)

5 Projects coordinated by Cyfronet, con t. 5 PLGrid NG Project ( ) (run by PL-Grid Consortium) Budget: total ca. 3,6 M Expected outcome: Extension of domain specific solutions by 14 add l domains Extension of resources and services by: ca. 8 Tflops, some PB

6 Focus on Users 6 Computer centres Hardware/Software User friendly Services Help Desk Domain Experts QoS/SLM Grants Real Users

7 TOP500 June 2011 Nov June 2013 Nov Polish Sites 7 Rank Site System Cores Rmax (TFlop/s) Rpeak (TFlop/s) Cyfronet Poland Zeus - Cluster Platform SL390/BL2x220, Xeon X5650 6C, 2.660GHz, Infiniband QDR, NVIDIA 2090 Hewlett-Packard 11,694 23,932 25,468 25, ICM Warsaw Poland BlueGene/Q, Power BQC 16C 1.600GHz, Custom Interconnect IBM 16,384 16,384 16, TASK Gdańsk GALERA PLUS -- Action Xeon HP BL 2x220/BL490 E5345/L5640 Infiniband ACTION 10, WCSS Wroclaw Cluster Platform 3000 BL2x220, X56xx, 2.66 GHz, Infiniband Hewlett-Packard 6, PCSS Poland Rackable C1103-G15, Opteron C 2.40 GHz, Infiniband QDR SGI 9,

8 Supercomputer Zeus 8 Xeon, 23 TB, 169 TFlops Opteron, 26 TB, 61 TFlops Xeon, 3,6 TB, 136 TFlops Xeon, 6 TB, 8 TFlops ZEUS Statistics 2012 (2013) Users needs taken into account Almost 8 mln jobs 21,000+ daily 80 mln CPU hours 9130 years 800+ active users 100PB+ usage of scratch (350PB) The longest job: 76 days The biggest job: 576 cores (1024) Ca. 50% CPU time for multicore jobs

9 Current Users of PL-Grid infrastructure 9 Number of users/employees Zmiana liczby aktywnych Użytkowników i Pracowników Infrastruktury - 32 miesiące Liczba Użytkowników Total number of Users Data Użytkownicy Infrastruktury Pracownicy Wszystkie konta zarejestrowane w Infrastrukturze

10 PLGrid Plus: Activities in general 10 Integration Services National and International levels Dedicated Portals and Environments Unification of distributed Databases Virtual Laboratories Remote Visualization Service value = utility + warranty SLA management Computing Intensive Solutions Specific Computing Environments Adoption of suitable algorithms and solutions Workflows Cloud computing Porting Scientific Packages Data Intensive Computing Access to distributed Scientific Databases Homogeneous access to distributed data Big Data Data discovery, process, visualization, validation. 4th Paradigm of scientific research Instruments in Grid Remote Transparent Access to instruments Sensor networks Organizational Organizational backbone Professional support for specific disciplines and topics Rich international collaboration (EGI.eu, EGI INSPIRE, EMI, PRACE..

11 Domain Grids 11 Program for strategic science domains and important topics of Polish/European Science Access to the software packages is provided by: glite Unicore QCG (usage : ca cores/month) (usage 2013: ca cores/month) (usage H2 2013: ca cores/month) local Already identified 13 communities/scientific topics: Astrophysics High Energy Physics Life Sciences Quantum Chemistry and Molecular Physics Synchrotron Radiation Energy Sector Metallurgy Nanotechnology Acoustics Ecology Bioinformatics Health Material Science

12 PLGrid NG New Generation of PLGrid 12 PLGrid NG ( ) (for the whole Consorium) IT experts and field experts Additional/new groups of experts involved Identified 14 communities/scientific topics: Medicine OpenOxides Mathematics Biology Hydrology Geoinformatics Meteorology Complex Networks ebaltic-grid UNRES VPH Computational Chemistry Nuclear Power Metal Processing

13 13 Examples of domain specific solutions and services (PLGrid PLUS use cases)

14 Deployed domain specific services Acoustics 14 Project Database the service is a web portal designed to provide PL-Grid users with the numerical database (simulation results) and experimental database (measurements results). Noise maps the service is an application used to generate noise maps in urban environments, based on data provided by the user. Animation: calculated dynamic noise map

15 Acoustics Noise Map service 15 The Noise Map service for creating maps of noise threats for roads, railways and industrial sources. Integration of the software service with the network of distributed sensors brings a possibility of making automatic updates of noise maps for a specified time period. Illustration of an application of the developed solution is the urban area noise mapping. The map can be updated completely within relatively short period of time, employing the PL-Grid Infrastructure. Operations are performed employing a dedicated noise prediction model, optimized for a computer cluster. In addition, predicted maps may be adjusted, using real noise level measurements. Animation: the fragment of the calculated dynamic noise map of the city of Gdansk, Poland

16 Metallurgy Simulations of extrusion process in 3D deployed 16 Main Objectives: Optimization of the metallurgical process of profiles extrusion. Optimization includes: shape of foramera, channel position on a die, calibration stripes, extrusion velocity, ingot temperatures, tools. The proposed grid-based software simulates extrusion of thin profiles and rods of special alloys of magnesium, containing calcium supplements. These alloys are characterized by extremely low technological plasticity during metal forming. The FEM mathematical model developed.

17 Metallurgy Deployed domain specific services 17 SSRVE the service for generation a Statistically Representative Volume Element (called SSRVE) for the microstructures of materials. Representation of microstructure of two-phase metallic materials ModFemNet the service for welding simulations.

18 Chemistry Deployed domain specific service 18 InSilicoLab for Chemistry the service that aims to support the launch of complex computational quantum chemistry experiments in the PL-Grid Infrastructure. Experiments of this service make it easy to plan sequential computation schemes that require the preparation of series of data files, based on a common schema. Animation showing the service run

19 Energy Sector πesa - Platform for Integrated Energy System Analysis Service to be deployed in 2014 πesa narzędzie do budowy modeli systemów energetycznych TIMES, system modelowania jakości powietrza Polyphemus oraz model do oceny ich oddziaływania na środowisko i zdrowie ludzkie MAEH. OptiMINE analiza wariantów prowadzenia robót górniczych i wybór najlepszego z nich pod względem planowanego wydobycia węgla kamiennego w kopalni. Algorytm obliczeniowy opiera się na selekcji klonalnej i innych wybranych elementach sztucznych systemów immunologicznych. Wartości postępów w poszczególnych wyrobiskach i rozłożenie robót w czasie, których dotrzymanie zapewni osiągnięcie planowanego wydobycia. ModWELL Video: Atmosphere transport of air pollution Analiza funkcjonowania sektora wytwarzania energii elektrycznej w horyzoncie krótkoterminowym w wysokiej rozdzielczości. Z pomocą Wirtualnego Laboratorium GridSpace 2 19

20 Work-in-Progress on PL-Grid Infrastructure 20 Unified Portal SSO Community Portal Community Portal Community Portal Accounting Unified Acess Portals Helpdesk Large-scale file systems Large-scale databases Cloud Services Computational Grids Computation and Data Processing Services CYFRONET ICM PCSS TASK WCSS Geographically Distributed Infrastructure

21 Conclusions 21 Further developement needed, as identified currently, mainly on Domain Specific Grids Request from the users communities Capacity for organization of future development according to Expertise and experience Strong scientific potential of the users communities being represented by PL-Grid Consortium Wide international cooperation concerning the Consortium and individual Partners, good recognition worldwide Good managerial capacity Please visit our Web pages: Credits

22 Credits Structural Funds. and people 22 ACC Cyfronet AGH Kazimierz Wiatr Michał Turała Łukasz Dutka Tomasz Szepieniec Mariusz Sterzel Robert Pająk Marian Bubak Krzysztof Zieliński Karol Krawentek Agnieszka Szymańska Maciej Twardy Teresa Ozga Angelika Zaleska-Walterbach Andrzej Oziębło Zofia Mosurska Marcin Radecki Renata Słota Tomasz Gubała Darin Nikolow Aleksandra Pałuk Patryk Lasoń Marek Magryś Łukasz Flis and many others domain experts. ICM Marek Niezgódka Piotr Bała Maciej Filocha PCSS Maciej Stroiński Norbert Meyer Krzysztof Kurowski Bartek Palak Tomasz Piontek Dawid Szejnfeld Paweł Wolniewicz WCSS Józef Janyszek Mateusz Tykierko Paweł Dziekoński Bartłomiej Balcerek TASK Rafał Tylman Mścislaw Nakonieczny Jarosław Rybicki