March 14th at the IBM Finland HQ in Helsinki, we are inviting around 100 people interested in industry-specific opentech AI, from AI for healthcare, retail, finance, energy, IoT, manufacturing, education, and other application areas. More coming soon…
Please email Jim Spohrer <email@example.com> and/or Daniel Pakkala <Daniel.Pakkala@vtt.fi> if you are interested in learning more about the workshop. We are looking for collaborators who know of open AI challenge leaderboards or capabilities that should be discussed at the Opentech AI Workshop. A leaderboard is a public website that provides information about an entities standing or ranking, with respect to a specific best effort attempt to solve a challenge or task. Open leaderboards advance science, since they allow others to replicate the result by providing access to code, data, and compute resources.
Another way of formulating the question is: Where can AI be applied in an industry specific manner (a task with open access data and code) to benchmark and to improve industry standard performance, and grow more opportunities for value creation? First understanding where leaderboards already exist is important (e.g., Wikipedia Question Answering), and second understanding where to create new leeaderboads to help drive progress is an AI-era methodology to master (e.g., Breast Cancer Tumor Proliferation Challenge). “Leaderboard solvers” are like the hammers, and where government and industry-leaders create strategically placed “leaderboard nails” is a methodology to master. This methodology will become increasingly important to master, as the pace of AI progress advances, driven in part by Moore’s Law (increasing compute power to solve classes of problems), zettabytes of data (the new oil), AI-related model marketplaces (Bot Asset Exchange, Acumos, etc.), algorithms (deep learning and reasoning), open source code (TensorFlow/Keras, Caffe, PyTorch, etc. on Github), commercialization-friendly licenses (Apache 2.0), and open governance (Linux Foundation, Apache Foundation, etc.). These are some factors combining today that organizations need to better understand.
Opportunities for AI challenge leaderboards include: AI Progress Metrics as mapped by Electronic Frontiers Foundation, AI Safety and other Industry Focus Areas as mapped by Partnership on AI, AI Leadership Diversity and Inclusiveness as mapped by AI4All, AI Activities and Progress as mapped by AI Index Report, etc.
Scroll down here to see industry-specific examples of AI challenge leaderboards: https://opentechai.blog/applications/