In 2025, we managed a significant volume of equipment procurement while simultaneously we prepare our server rooms for these new IT servers. A key milestone was the construction of a dedicated IT data center where a new automated tape library has been installed. To ensure the equipment’s optimal performance, this facility features a precision humidity control system.
As a result of these investments, our current operational capacity is distributed across our services as follows:
Storage Solutions We have significantly increased our long-term archiving and high-performance storage capabilities:
Tape Libraries (Backup & Archiving):
dCache System: Currently managing 52PB of net space for high-end data storage.
Ceph Platform: A robust HDD-based architecture with a total of 4.2PB, allocated as follows:
Computing Resources Our processing power has been scaled to meet increasing computational demands:
CosmoHub
In 2025, significant activity took place around the CosmoHub science portal and its underlying Hadoop platform.
Most notably, 13 years after its initial release, a new version of the CosmoHub frontend was deployed. This new version was fully rewritten using a modern technology stack and features a refreshed user interface together with several new functionalities (see the corresponding Release Note for further details).
PIC continues to maintain a strong commitment to the implementation of open standards, particularly those defined by the International Virtual Observatory Alliance (IVOA). During 2025, substantial effort was dedicated to advancing this objective within the CosmoHub ecosystem. A new and fully refactored API was developed, incorporating modern development best practices such as cooperative multitasking in order to improve performance and stability. This work lays the foundation for the deployment of a broad set of features aimed at enhancing the interoperability and usability of the scientific platform.
In this context, a WebDAV door to PIC’s Hadoop Distributed File System (HDFS) cluster (webdav-hdfs.pic.es) was deployed. The system was implemented as a new WebHDFS provider within the AsgiWebDAV package and contributed back to the open-source community. In addition, development of a VOTable endpoint on top of the WebDAV-HDFS door was initiated, and CosmoHub’s compliance with the ADQL standard was reviewed.
Two complementary projects within the CosmoHub/Hadoop ecosystem were successfully carried out through student internships:
CosmoHub Chat: development of a Retrieval-Augmented Generation (RAG) chatbot designed to assist users in generating SQL queries for integration within CosmoHub.
dCache Billing Streaming: implementation of an ETL pipeline that reads dCache operational logs from Kafka and ingests them into Hive tables to enable advanced analytics.
Authentication and Authorization (A&A)
PIC has undertaken a major modernization of its user identity and access management infrastructure. In particular, a fully automated onboarding system was deployed to seamlessly integrate new user registrations with PIC’s internal directory services.
Through a centralized authorization framework, collaborating researchers can now automatically obtain the appropriate access privileges by authenticating through their home institutions’ identity systems. To ensure strict security controls, user access rights are dynamically verified and updated at every login session, preventing unauthorized users from accessing protected scientific computing resources.
Furthermore, the center has streamlined its previously manual, sponsor-based approval workflows and introduced enhanced auditing capabilities to improve traceability of administrative actions within research groups. This new architecture also establishes the foundation required to integrate all CosmoHub users into PIC’s core computing environment in the near future.
Rucio
Throughout 2025, PIC reinforced its commitment to Rucio as the primary technology for large-scale data management. The team maintained an active presence within the global Rucio community, ensuring the stability and performance of local instances through consistent maintenance and strategic upgrades. Furthermore, PIC sustained its leadership in the development of the Rucio JupyterLab extension.
During the past year, the group has continued to consolidate its research activities and training efforts. The team currently includes four PhD students in progress, one of whom joined the group in 2025. In addition to supervising doctoral research, the group has been actively involved in the execution of the Spanish Plan Nacional project (AYA) led by the PI, advancing the planned scientific objectives. Within this framework, both the PI and Jiefeng joined the ARRAKHIS collaboration and initiated research on AI-based deblurring techniques for astronomical imaging
The group has also strengthened its interdisciplinary activities in materials science. In this context, we collaborated on a review “Applied Artificial Intelligence in Materials Science and Material Design”. The PhD student J.Bueno from ICMM visited PIC for a month to collaborate on a project on designing metasurfaces using diffusion models. Collaborating with ALBA and Eurecat, we presented a poster in AI4AM on analyzing EELS spectra from 3rd generation Mn steel. With ICN2 we published a paper on denoising electron microscopy in the frequency domain.
The group also contributed to training activities through the organization of an informal, hands-on course on deep learning at PIC. The course is structured in small, interactive groups and aims to provide participants with practical experience in training and understanding neural networks, with the goal of enabling them to apply these techniques to their own research projects. The material spans from fundamental concepts to more advanced topics such as generative AI and diffusion models. An edition of the course took place in spring 2025, followed by a substantially updated edition starting in November 2025, including revised introductory material and a stronger focus on participant-led research projects. The course is held weekly at PIC and brings together participants from several institutions, including PIC, IFAE, ICN2, ICFO, ALBA, and other research groups working in areas such as astrophysics and materials science.
In 2025, the pledges for the Spanish Tier-1 center hosted at PIC included approximately 165 kHS23 (equivalent to 13,000 CPU cores), 19 PB of disk storage, and 54 PB of tape storage. Beyond providing substantial computing and storage capacity, a defining feature of a Tier-1 center is its ability to deliver these resources through highly reliable, 24/7 operations. Additionally, PIC hosts 25% of the Spanish ATLAS Tier-2 federation, accounting for another 22.5 kHS23 (1,700 CPU cores) and 2.5 PB of disk storage. Demonstrating outstanding operational excellence, PIC ranked at the top of the WLCG’s stability and reliability metrics in 2025 for both its Tier-1 and Tier-2 components.
Beyond providing essential resources to the WLCG, the PIC team actively drives R&D initiatives across multiple LHC experiments. These efforts are vital for scaling the infrastructure to meet the increasing demands of the LHC scientific programme and preparing for the High-Luminosity LHC (HL-LHC) phase. The team’s significant contributions to computing conferences and publications highlight their focus on key innovations: leveraging HPC resources at the Barcelona Supercomputing Center (BSC), optimizing the use of MareNostrum 5, developing next-generation services for Analysis Facilities, and evaluating data caches for the WLCG. Furthermore, the team played a pivotal role in drafting the ATLAS Computing TDR and CMS Computing CDR, the foundational technical reports that define the R&D milestones leading into the HL-LHC era. Beyond technical development, PIC provides crucial strategic leadership, actively shaping future infrastructure through its contributions to the WLCG Technical Coordination Board, the HEPIX community leadership, and by leading the WLCG Job Allocations and Handling working group, as well as within the ATLAS and CMS Computing management boards.
Sustainability remains a key strategic focus at PIC, positioning the center at the forefront of green computing in high-energy physics. To align compute farm usage with external factors, such as green energy availability and off-peak electricity prices, we have successfully characterized PIC’s CPUs. This foundational work measured the impact of adjusting CPU clock frequencies to optimize energy efficiency. The next phase of this pioneering initiative involves automating these processes. By developing machine learning and AI models for predictive power scaling, we aim to dynamically adapt the system, intelligently identifying optimal time windows to adjust frequencies (or even drain compute nodes filled with short jobs), ultimately reducing the facility’s overall CO₂ footprint.
As the HL-LHC era approaches in 2030, WLCG computing centers must be prepared to support unprecedented wide-area network (WAN) throughputs. To meet these escalating demands, the PIC team has worked closely with CSUC and RedIRIS to secure a major infrastructure upgrade, expanding the center’s network to a 400 Gbps symmetric connection in 2026, with further expansions targeted for 2029. To validate readiness for the HL-LHC, the WLCG conducts a series of rigorous, large-scale Data Challenges. PIC has already proven the resilience of its infrastructure and services by successfully passing the 2024 Data Challenge (DC24), which evaluated Tier-1 sites at 25% of total HL-LHC requirements. Now, the confirmed 400 Gbps symmetric upgrade perfectly positions PIC to meet the demands of the upcoming DC27, ensuring robust and scalable connectivity for the research community.
Further demonstrating its leadership within the global high-energy physics computing community, the PIC team spearheaded the organization of several premier international technical events throughout 2025. The team played a leading role in organizing the HEPIX Spring Workshop in Lugano (Switzerland), the HEPIX Autumn Workshop in Lanzhou (China), and the European HTCondor Workshop in Prague (Czech Republic). Most notably, PIC locally organized the FTS & XRootD Workshop in October 2025, bringing this critical data management event directly to PIC and the local campus at the Universitat Autònoma de Barcelona (UAB). The annual CMS Offline Software and Computing management Face to Face meetings were also hosted by the PIC team at the UAB in June 2025. The researchers also participated in the organization and mentoring students in the summer school in experimental physics Phy6cool 2025, held in June 2025 at CIEMAT, Madrid.
P2025 saw the conclusion of the In-situ Correlative Facility for Advanced Energy Materials (InCAEM) project. The Electron Microscope (EM) has been installed at ALBA, and its commissioning is expected to end by mid-2026. Following the planning and initial purchases of 2024, the full hardware infrastructure for online and offline data processing was acquired and installed in 2025, alongside the development of the necessary software framework. The hardware acquired and deployed at PIC to support the project includes:
In terms of software, a unidirectional data synchronization service to transfer data from ALBA facilities to PIC was deployed and tested, ensuring the infrastructure can handle the expected data rates of the project. Additionally, developments in Authentication and Authorization (A&A) will facilitate the onboarding of InCAEM users wanting to perform correlative studies at PIC. Beyond infrastructure development, PIC researchers contributed to community initiatives in AI for advanced microscopy. The team participated in the organization of the AMATs conference, fostering interdisciplinary exchanges between microscopy, materials science, and artificial intelligence.
PIC also co-organized the local node of the AI for Microscopy Hackathon, an international event focused on the development of machine learning methods for microscopy data analysis. In this context, PIC researchers participated with the submission “Latent-Constrained Autoencoder for 4D-STEM Clustering”, developed jointly with collaborators from ALBA and ICN2. The work explored representation learning techniques for the analysis and clustering of high-dimensional 4D-STEM datasets.