1 Domain-oriented services and resources of Polish Infrastructure for Supporting Computational Science in the European Research Space Structural Funds in Poland Polish NGI and Cyfronet case study Jacek Kitowski ACK Cyfronet AGH, Krakow, Poland PL-Grid Consortium EC Workshop Structural Funds, Brussels, 6.2.2014
Outline 2 Motivation and Background Consortium Family of PL-Grid Projects Domain-specific solutions and services Overview Use-cases Expertise Conclusions
Motivation and Background 3 Computational Science problems to be addressed: (initiative from scientific community) Algoritms, environments and deployment 4 th paradigm, Big Data, Data Farming Domain-Specific solutions and environments PL-Grid Consortium (2007) 5 Polish Computer Centres Continuous development based on: Projects funded by the European Regional Development Fund as part of the Innovative Economy Program Polish Optical Internet (PIONIER) (2x10+ Gb/s) Close international collaboration (EGEE, EGI InSPIRE, PRACE, ESFRI projects, e.g., CTA, EPOS, ERIC cases..) RoadMaps for e-infrastructure developments Previous projects (5FP, 6FP, 7FP, EDA ) Integration with international platforms
Projects coordinated by Cyfronet 4 PL-Grid Project (2009 2012) (run by PL-Grid Consortium) Budget: total 21 M, from EU 17M Number of people involved: ca. 80 (total, from different Polish Centres) Outcome: Common base infrastructure National Grid Infrastructure (NGI_PL) Resources: 230 Tflops, 3.6 PB PLGrid PLUS Project (2011 2015) (run by PL-Grid Consortium) Budget: total ca.18 M, from EU: ca.15 M Number of people involved: ca. 120 IT scientists Computational scientists Field experts Expected outcome: Focus on users Domain specific solutions: 13 domains (Specific computing environments) QoS by SLM Extension of resources and services by: 500 Tflops, 4.4 PB Keeping diversity for users Clusters (thin and thick nodes, GPU), SMP, vsmp Clouds (OpenStack, Open Nebula)
Projects coordinated by Cyfronet, con t. 5 PLGrid NG Project (2014 2015) (run by PL-Grid Consortium) Budget: total ca. 3,6 M Expected outcome: Extension of domain specific solutions by 14 add l domains Extension of resources and services by: ca. 8 Tflops, some PB
Focus on Users 6 Computer centres Hardware/Software User friendly Services Help Desk Domain Experts QoS/SLM Grants Real Users
TOP500 June 2011 Nov. 2012 June 2013 Nov. 2013 Polish Sites 7 Rank Site System Cores Rmax (TFlop/s) Rpeak (TFlop/s) 81 106 113 145 Cyfronet Poland Zeus - Cluster Platform SL390/BL2x220, Xeon X5650 6C, 2.660GHz, Infiniband QDR, NVIDIA 2090 Hewlett-Packard 11,694 23,932 25,468 25,468 104.8 234.3 266.9 266.9 124.4 357.5 373.9 373.9 143 170 221 ICM Warsaw Poland BlueGene/Q, Power BQC 16C 1.600GHz, Custom Interconnect IBM 16,384 16,384 16,384 172.7 189.0 189.0 209.7 209.7 209.7 163 TASK Gdańsk GALERA PLUS -- Action Xeon HP BL 2x220/BL490 E5345/L5640 Infiniband ACTION 10,384 65.6 97.8 194 WCSS Wroclaw Cluster Platform 3000 BL2x220, X56xx, 2.66 GHz, Infiniband Hewlett-Packard 6,348 57.4 67.5 375 PCSS Poland Rackable C1103-G15, Opteron 6234 12C 2.40 GHz, Infiniband QDR SGI 9,498 89.8 211.1
Supercomputer Zeus 8 Xeon, 23 TB, 169 TFlops Opteron, 26 TB, 61 TFlops Xeon, 3,6 TB, 136 TFlops Xeon, 6 TB, 8 TFlops ZEUS Statistics 2012 (2013) Users needs taken into account Almost 8 mln jobs 21,000+ daily 80 mln CPU hours 9130 years 800+ active users 100PB+ usage of scratch (350PB) The longest job: 76 days The biggest job: 576 cores (1024) Ca. 50% CPU time for multicore jobs
Current Users of PL-Grid infrastructure 9 Number of users/employees Zmiana liczby aktywnych Użytkowników i Pracowników Infrastruktury - 32 miesiące 2 500 2 000 1847 1882 1902 1949 Liczba Użytkowników 1 500 1 000 500 391 412 452 512 570 607 628 637 648 Total number of Users 782 859 878 885 924 971 998 1 0541 088 1 106 1 131 1 169 1 402 1 4361 4711 534 1 3601 376 1 333 1 3511 363 1 396 1 413 1 459 140 144 149 150 153 157 159 159 161 161 164 164 167 168 179 183 205 221 231 237 241 225 228 227 230 236 238 232 233 236 239 238 241 0 1.02.2011 1.05.2011 1.08.2011 1.11.2011 1.02.2012 1.05.2012 1.08.2012 1.11.2012 1.02.2013 1.05.2013 1.08.2013 Data Użytkownicy Infrastruktury Pracownicy Wszystkie konta zarejestrowane w Infrastrukturze
PLGrid Plus: Activities in general 10 Integration Services National and International levels Dedicated Portals and Environments Unification of distributed Databases Virtual Laboratories Remote Visualization Service value = utility + warranty SLA management Computing Intensive Solutions Specific Computing Environments Adoption of suitable algorithms and solutions Workflows Cloud computing Porting Scientific Packages Data Intensive Computing Access to distributed Scientific Databases Homogeneous access to distributed data Big Data Data discovery, process, visualization, validation. 4th Paradigm of scientific research Instruments in Grid Remote Transparent Access to instruments Sensor networks Organizational Organizational backbone Professional support for specific disciplines and topics Rich international collaboration (EGI.eu, EGI INSPIRE, EMI, PRACE..
Domain Grids 11 Program for strategic science domains and important topics of Polish/European Science Access to the software packages is provided by: glite Unicore QCG (usage 2011-2013: ca. 3000 cores/month) (usage 2013: ca. 60-100 cores/month) (usage H2 2013: ca. 2300 cores/month) local Already identified 13 communities/scientific topics: Astrophysics High Energy Physics Life Sciences Quantum Chemistry and Molecular Physics Synchrotron Radiation Energy Sector Metallurgy Nanotechnology Acoustics Ecology Bioinformatics Health Material Science
PLGrid NG New Generation of PLGrid 12 PLGrid NG (2014-2015) (for the whole Consorium) IT experts and field experts Additional/new groups of experts involved Identified 14 communities/scientific topics: Medicine OpenOxides Mathematics Biology Hydrology Geoinformatics Meteorology Complex Networks ebaltic-grid UNRES VPH Computational Chemistry Nuclear Power Metal Processing
13 Examples of domain specific solutions and services (PLGrid PLUS use cases)
Deployed domain specific services Acoustics 14 Project Database the service is a web portal designed to provide PL-Grid users with the numerical database (simulation results) and experimental database (measurements results). Noise maps the service is an application used to generate noise maps in urban environments, based on data provided by the user. Animation: calculated dynamic noise map
Acoustics Noise Map service 15 The Noise Map service for creating maps of noise threats for roads, railways and industrial sources. Integration of the software service with the network of distributed sensors brings a possibility of making automatic updates of noise maps for a specified time period. Illustration of an application of the developed solution is the urban area noise mapping. The map can be updated completely within relatively short period of time, employing the PL-Grid Infrastructure. Operations are performed employing a dedicated noise prediction model, optimized for a computer cluster. In addition, predicted maps may be adjusted, using real noise level measurements. Animation: the fragment of the calculated dynamic noise map of the city of Gdansk, Poland
Metallurgy Simulations of extrusion process in 3D deployed 16 Main Objectives: Optimization of the metallurgical process of profiles extrusion. Optimization includes: shape of foramera, channel position on a die, calibration stripes, extrusion velocity, ingot temperatures, tools. The proposed grid-based software simulates extrusion of thin profiles and rods of special alloys of magnesium, containing calcium supplements. These alloys are characterized by extremely low technological plasticity during metal forming. The FEM mathematical model developed.
Metallurgy Deployed domain specific services 17 SSRVE the service for generation a Statistically Representative Volume Element (called SSRVE) for the microstructures of materials. Representation of microstructure of two-phase metallic materials ModFemNet the service for welding simulations.
Chemistry Deployed domain specific service 18 InSilicoLab for Chemistry the service that aims to support the launch of complex computational quantum chemistry experiments in the PL-Grid Infrastructure. Experiments of this service make it easy to plan sequential computation schemes that require the preparation of series of data files, based on a common schema. Animation showing the service run
Energy Sector πesa - Platform for Integrated Energy System Analysis Service to be deployed in 2014 πesa narzędzie do budowy modeli systemów energetycznych TIMES, system modelowania jakości powietrza Polyphemus oraz model do oceny ich oddziaływania na środowisko i zdrowie ludzkie MAEH. OptiMINE analiza wariantów prowadzenia robót górniczych i wybór najlepszego z nich pod względem planowanego wydobycia węgla kamiennego w kopalni. Algorytm obliczeniowy opiera się na selekcji klonalnej i innych wybranych elementach sztucznych systemów immunologicznych. Wartości postępów w poszczególnych wyrobiskach i rozłożenie robót w czasie, których dotrzymanie zapewni osiągnięcie planowanego wydobycia. ModWELL Video: Atmosphere transport of air pollution Analiza funkcjonowania sektora wytwarzania energii elektrycznej w horyzoncie krótkoterminowym w wysokiej rozdzielczości. Z pomocą Wirtualnego Laboratorium GridSpace 2 19
Work-in-Progress on PL-Grid Infrastructure 20 Unified Portal SSO Community Portal Community Portal Community Portal Accounting Unified Acess Portals Helpdesk Large-scale file systems Large-scale databases Cloud Services Computational Grids Computation and Data Processing Services CYFRONET ICM PCSS TASK WCSS Geographically Distributed Infrastructure
Conclusions 21 Further developement needed, as identified currently, mainly on Domain Specific Grids Request from the users communities Capacity for organization of future development according to Expertise and experience Strong scientific potential of the users communities being represented by PL-Grid Consortium Wide international cooperation concerning the Consortium and individual Partners, good recognition worldwide Good managerial capacity Please visit our Web pages: http://www.plgrid.pl/en http://www.plgrid.pl Credits
Credits Structural Funds. and people 22 ACC Cyfronet AGH Kazimierz Wiatr Michał Turała Łukasz Dutka Tomasz Szepieniec Mariusz Sterzel Robert Pająk Marian Bubak Krzysztof Zieliński Karol Krawentek Agnieszka Szymańska Maciej Twardy Teresa Ozga Angelika Zaleska-Walterbach Andrzej Oziębło Zofia Mosurska Marcin Radecki Renata Słota Tomasz Gubała Darin Nikolow Aleksandra Pałuk Patryk Lasoń Marek Magryś Łukasz Flis and many others domain experts. ICM Marek Niezgódka Piotr Bała Maciej Filocha PCSS Maciej Stroiński Norbert Meyer Krzysztof Kurowski Bartek Palak Tomasz Piontek Dawid Szejnfeld Paweł Wolniewicz WCSS Józef Janyszek Mateusz Tykierko Paweł Dziekoński Bartłomiej Balcerek TASK Rafał Tylman Mścislaw Nakonieczny Jarosław Rybicki