Associate Professor at the Technical University of Valencia (UPV) and former Scientific Project Officer at the Joint Research Centre, European Commission. Part of VRAIN, ELP and DMIP Team. Director of the Master in Big Data Analytics at the UPV. Teaching in the Bachelor’s Degree in Informatics Engineering (UPV), Interactive Technologies (EPSG), Master’s Degree in Informatics Engineering (UPV), Master’s in Business Administration and Digital Transformation (UPV), and the Master’s Degree in Artificial Intelligence Research (UIMP). I received a B.Sc. degree in Computer Science (2009), M.Sc. degree in Software Engineering, Formal Methods and Information Systems (2010), a Post-Graduate Diploma in Engineering Business Management (2011) and a Ph.D in Computer Science (2016) from UPV. The evaluation and measurement of intelligent systems is, among other more applied research topics, the main scientific-technical objective of my research agenda.
Recent Highlights
Subscribe to our AI Evaluation Digest for the latest updates!
Area Chair for ECAI 2024, join us in Santiago de Compostela!
Presented our paper “Your Prompt is My Command: On Assessing the Human-Centred Generality of Multimodal Models” in AAAI 2024.
Paper accepted in IDA: Cracking black-box models: Revealing hidden machine learning techniques behind their predictions
Member of the OECD Expert Group on AI Futures
Paper accepted for ECAI 2023: Adversarial Benchmark Evaluation Rectified by Controlling for Difficulty
Paper published in Science! “Rethink reporting of evaluation results in AI”
Co-organising a Workshop on Predictable AI
Two papers accepted for JAIR: “Your Prompt is My Command: On Assessing the Human-Centred Generality of Multimodal Models” and SUSCOM: ” Trends in AI Inference Energy Consumption: Beyond the Performance-vs-Parameter Laws of Deep Learning”
European Commission JRC Excellence Award: 2021 Best Young Researcher
Paper published in the Machine Learning Journal: “Can language models automate data wrangling?”
BigBench paper published! “Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models”
The European Commission’s “Glossary of human-centric artificial intelligence” it’s out there now!
Two papers accepted for AAAI 2022: Training on the Test Set: Mapping the System-Problem Space in AI (Blue Sky Idea Runner-Up Award!) and When AI Difficulty is Easy: The Explanatory Power of Predicting IRT Difficulty
Papers accepted for publication in the Nature Machine Intelligence journal: Research community dynamics behind popular AI benchmarks, Nature Scientific Reports General intelligence disentangled via a generality metric for natural and artificial intelligence, the Wiley International Journal of Intelligent Systems: “Missing the missing values: The ugly duckling of fairness in machine learning”, the Journal of Artificial Intelligence Research: “Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks”, and the Elsevier Telematics and Informatics: “Futures of Artificial Intelligence through Technology Readiness Levels”, 2021
Co-organising the 1st Workshop on Evaluating Progress in AI (EPAI2020) @ ECAI 2020 taking place (digitally) in Santiago (Spain) in September 4, 2020
3 papers accepted for ECAI 2020: Tracking AI: The Capability is (Not) Near, AI Paradigms and AI Safety: Mapping Artefacts and Techniques to Safety Issues and Family and Prejudice: A Behavioural Taxonomy of Machine Learning Techniques
Papers accepted for IEEE Transactions on Knowledge and Data Engineering journal: CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories, and the IEEE Transactions on Games Journal: Dual Indicators to Analyse AI Benchmarks: Difficulty, Discrimination, Ability and Generality, 2020
Paper accepted for the AAAI/ACM Conference on Artificcial Intelligence, Ethics, and Society: “Does AI Qualify for the Job? A Bidirectional Model Mapping Labour and AI Intensities”, 2020
Paper accepted for the Artificial Intelligence Journal: Item Response Theory in AI: Analysing Machine Learning Classifiers at the Instance Level, 2019
2 papers accepted for ECML 2019: Automated Data Transformation with Inductive Programming and Dynamic Background Knowledge and ABK-ADAPT: Dynamic Background Knowledge for Automating Data Transformation
Program Committees
- IJCAI 2019, 2020, 2022, 2024
- AAAI 2020, 2021, 2022+, 2023+, 2024+
- NIPS/NeurIPS 2018, 2019, 2020, 2021
- UAI 2019, 2020, 2021, 2022, 2023
- ICRL 2019, 2020, 2021
- ECAI 2020, 2024*
- ECML 2020, 2021, 2022, 2023, 2024
- ACM SIGKDD 2021
- ICML 2021, 2022, 2023
- AAMAS 2021, 2022, 2023, 2024
- ICMLA 2021, 2022, 2023, 2024
- ACL 2023
+: Senior PC member *: Area Chair
Organising Committees
- Co-organised the Workshop on Predictable AI.
- Co-organised the 1st Evaluation Beyond Metrics (EBeM 2022) Workshop at IJCAI 2022.
- Co-organised the 1st Evaluating Progress in AI (EPAI2020) Workshop at ECAI 2020.
- Co-organised the 1st Architectures and Evaluation for Generality, Autonomy & Progress in AI (AEGAP) Workshop at IJCAI 2018.
- Co-organised the 2nd Evaluating General-Purpose AI (EGPAI) Workshop at IJCAI 2017.
- Co-organised the 1st Data Wrangling Automation (DWA) Workshop at ICDM 2016.
Editorial Boards
Awards
AAAI 2022 Blue Sky Idea Runner-Up Award
UPV Research Awards 2022: Excellent Research Publication Award Runner-up
European Commission JRC Excellence Award: 2021 Best Young Researcher
Best Poster Award @ 16th IEEE ICCSE: Project-Based Learning for Scaffolding Data Scientists’ Skills
Best Paper Award @ ECAI 2016: Making sense of item response theory in machine learning
Best Paper Award Runner-up @ AAAI SafeAI 2019: Surveying Safety-relevant AI Characteristics
2nd Prize (Local) & 2nd Prize (National) VI Hackathon Telefónica HackForGood 2018 - Project ParticleAI.
1st prize II OpenDatathon ETSINF-UPV 2017: Valencia City council Award - Project bikeFlow - TechReport.
1st Prize (Local) & 3rd Prize (National) IV Hackathon Telefónica HackForGood 2016 - Project BikeXplorer.
1st prize I OpenDatathon ETSINF-UPV 2016: BigML Award for Best predictive project - Project TrafficXplorer.