The Standard Industries Chemical Innovation Challenge (the “SI Challenge” or “Challenge”) is an ambitious initiative sponsored by Standard Industries Inc. and its affiliates, in particular W. R. Grace & Co., aimed at transforming small-molecule synthesis through the integration of AI/ML-based tools. Its primary goal is to stimulate research in creating retrosynthesis tools imbued with chemical intuition. This involves leveraging reported chemical reactions and reactivity patterns to predict altogether new transformations and chemically-reasonable disconnects that have yet to be experimentally validated. By encouraging participants to develop models tailored to molecular challenges with widespread relevance, the SI Challenge aims to promote a new generation of researchers integrating AI into their work. This evolution has the potential to reshape experimental synthetic chemistry, affecting areas like drug discovery, material science, environmental sustainability, and other essential scientific domains.
Current Solutions and Their Shortcomings Since first proposed by E. J. Corey in 1977, multiple retrosynthetic frameworks and approaches have been developed. In the last two decades, computer-assisted retrosynthetic tools have been developed, both for commercial use and through academic efforts. While remarkable progress has been achieved, all approaches that we are aware of to date are limited in one or more of the following ways:
Trained on inadequate or inaccurate datasets.
Poor prioritization and/or tree algorithm, leading to multiple similar syntheses, synthetic “dead ends”, or highly inefficient/impractical routes.
Biased toward reactions that are commonly used in the training sets.
Requires Ph.D. chemists to build complex reaction “rules” to identify suitable compatibility of functional groups -or- proposes reactions where certain functional groups are known to be incompatible.
Forward and retro-synthesis looks one step ahead and one step behind.
Rules-based or hand-coded that make it difficult to train with new data.
Allow reaction pathways that are difficult to scale (>100Kg) or have sustainability disadvantages.
The focus of the SI Challenge can be stated simply, but has proven remarkably hard to do thus far: create a retrosynthetic tool that can propose novel chemistry and is able to display chemical intuition for the construction of synthetic routes to complex organic molecules.
Why This Challenge is Needed The goal of the SI Challenge is to develop, test, and advance an approach for retrosynthesis with durable chemical intuition using artificial intelligence/machine learning. If successful, this solution would be transformative for organic chemists, resulting in access to novel chemical space, more efficient syntheses, enhanced scalability, and enhanced sustainability through the reduction in the use of expensive or toxic reagents. More importantly, it could serve to construct a more effective tool for what has been largely an empirical endeavor for over two centuries.
Guidelines
The Challenge Winner will provide an AI/ML-driven research platform designed to identify multiple options for synthesizing a target molecule. The AI/ML platform will assign probabilities of success or confidence levels to each pathway based on how close it is to literature precedent. The AI/ML platform will propose pathways that include reaction steps that are not explicitly listed in the literature but are derived by “chemical intuition” or analogy to known molecular transformations. Preferably, the AI/ML platform will allow the user to prioritize pathways by excluding some types of reactions and show preference to other types of reactions. The AI/ML platform will provide a JSON output to allow streamlined judging. Preferably, the AI/ML platform will allow the user to exclude some types of expensive reagents and prioritize starting molecules of lower cost. Preferably, the AI/ML platform will allow the user to prioritize pathways based on the types of equipment necessary to perform the reaction steps in the pathway.
Evaluation Criteria - Justifications and Methodology Judges will evaluate Participants using multiple evaluation methods, including but not limited to: Screening, Benchmarking, Pass/Fail, and Scorecard. ‘Screening’ is criteria of minimum thresholds that must be met to qualify and are judged as ‘pass/fail.’ Benchmarking evaluates Participants against a defined ‘baseline.’ Pass/Fail identifies the minimum Criteria Participants must meet to advance. Finally, the scorecard Criteria ranks Participants alongside multiple Criteria. Points are accrued and totaled at the end of each phase and at the end of all phases to inform the results.
Eligibility for Challenge Participation Commercial Teams
Must be a company with no more than 30 employees at the time of entering the Challenge.
The company should not have raised funds beyond a Series A round as of the time of entering the Challenge.
Academic Teams
Must be affiliated with an accredited academic institution at the time of entering the Challenge.
Individuals and Independent Teams
Must be able to demonstrate the ability to execute on their proposed solution. Ability will be determined at the Sponsor’s discretion at the point of submission.
Individuals must be 18 years of age to enter the Contest.
General Exclusions
Individuals or entities located in a jurisdiction where participation in the Challenge is prohibited or restricted by law or regulation.
Individuals or entities with a residence in or who are a national of Cuba, Iran, Syria, North Korea, Sudan or, as applicable, Russia, Crimea and covered regions of Ukraine; or an entity or individual who is subject to export controls, embargos, or sanctions of the United States.
Individuals or entities currently engaged with or employed by Sponsor, or that have immediate family members currently engaged with or employed by Sponsor, or those working on a proof-of-concept project, and their immediate family members, spouse, or significant other. Exception to this exclusion may be made at Sponsor’s discretion.
Individuals and entities that work directly with Judges in an academic or professional context and their immediate family members, spouse or significant other.
Individuals or entities that have a business relationship or sponsored research affiliation with the pharmaceutical industry or contract manufacturing and development organizations will be determined as eligible or ineligible at Sponsor’s discretion based on the nature of the relationship to such organizations.
Individuals or entities that have or are currently serving as consultants for Sponsor are ineligible to participate. Exceptions to this exclusion may be made by Sponsor where Sponsor believes the consultant does not possess information that would provide an unfair advantage over other Participants.
All Submissions related to the contest must be in English. Submissions in any other language will not be considered.
More information may be found in the Complete Guidelines PDF located in the Resources Tab of this page.
Competition Phases
Please see the "Timeline" section for the key dates associated with each phase. Dates are subject to change and will be updated as necessary over the course of the competition.
Phase 1:White Paper Phase Teams will be asked to submit technical proposals, or white papers, detailing the following:
Overall approach to chemical intuition in a retrosynthesis pathway, including limitations of the approach that might impact generalizability.
Proposed initial training dataset.
Description of feasibility, defining AI/ML approach, chemical and reaction descriptors, and descriptions of limitations of feasibility.
Participants acknowledge, by submitting their White Paper, that any information submitted during the White Paper Phase is not considered confidential information. The White Paper, and all Submissions provided by Participants during the Challenge, will be evaluated by a panel of judges, comprising an interdisciplinary team of subject-matter experts in competition-related fields.
Phase 2:Semifinalist Phase Up to 10 of the most promising proposals from the White Paper phase will advance to develop Proof of Concept models. To participate in this phase, participants will be required to sign a 2-Way NDA to allow for more detailed sharing about their strategy and results. During this phase:
Semifinalists will advance their approach with a set of ten (10) or more molecules provided by Sponsor (“Test Molecules”). Semifinalists will advance their approach to: i) identify key transformations for the total synthesis of the Test Molecules; and ii) propose novel approaches to those transformations or the broader total synthesis.
Semifinalists will construct a rudimentary interface so that real time evaluation of Test Molecules can be evaluated.
A confidence score will be defined by each Semifinalist and included as an output of any reaction step.
Semifinalists can evaluate additional molecules of their choice to better demonstrate their solution’s effectiveness, though this is not mandatory.
To reduce the influence of training data, the Challenge Sponsor may provide each Semifinalist with a training set. Each Semifinalist may augment this data with additional datasets.
At the end of the Semifinalist Phase, each Semifinalist will present its progress to the Judges Panel. Sponsor will provide several additional Test Molecules to Semifinalists at this time to demonstrate their solution in real-time. The Judges Panel will then select up to three (3) Semifinalists (“Finalists”) judged as having the solutions that best meet the Challenge criteria to continue to the Finalist Phase.
Phase 3: Finalist Phase The top 3 teams will sign a Business Agreement detailing terms of the relationship between participant and Standard Industries, such as exclusivity and other business terms. In this final phase:
Sponsor will provide each Finalist with five or more Challenge Molecules. Finalists will have an opportunity to further refine their approach and solution with these five Challenge Molecules.
Finalists will present their solution’s output for each of the five Challenge Molecules, defining the key transformations and the proposed synthetic routes.
Using appropriate intermediates/starting materials, Sponsor or a third party designated by Sponsor may attempt the key predicted transformation(s).
Based on the Challenge evaluation criteria, including real world results of the predictive model, the Judging Panel will select a Challenge Winner from among the Finalists to receive the Grand Award.
The Challenge Winner will adhere to the milestones set by the Challenge Sponsor and will provide updates on their progress..
To participate in the Finalist Phase, Finalists must acknowledge and accept additional terms detailing the desired relationship between a Finalist selected as the Challenge Winner (the Participant with the solution judged to best meet the overall Challenge evaluation criteria) and Sponsor (These terms are set out in the Challenge T&Cs and are binding on all Finalists accepting the Finalist award and become effective on the Challenge Winner when the Winner accepts the Grand Award set out below).
Judging Criteria
Phase 1: White Paper Phase White Papers will be screened for eligibility and completeness on a rolling basis during the submission period by the Sponsor. In the event that prospective participants are not eligible, they will be notified. Incompleteness is defined as either a missing field on the White Paper form, or a field with insufficient detail. Prospective Participants will be notified if one or more fields are missing or if the Sponsor have determined one or more fields have insufficient detail. Prospective Participants will be given an opportunity to resubmit their White Paper. If Prospective Participants do not resubmit or if the Sponsor determines that there is still insufficient detail in one or more fields, the White Paper will not progress to Content Review by the Judges as described below. White Paper proposals will be assessed and quantifiably measured through specific KPIs as detailed in the table below:
Criteria
Description
Weight
Completeness
Does the approach, either using prior work or a novel approach, encompass a complete solution?
10%
Methodology
Is the approach methodologically sound? Ambitious proposals are strongly encouraged; however, the methodology must be plausible and computationally achievable.
30%
Team & Resources
Does the team have the expertise and computational resources to develop a solution within the competition timeframe? Has the team done prior work that de-risks the proposed approach?
20%
Impact
Is it likely that, if successful, the approach will have a meaningful impact on the field?
30%
Innovation
How does the approach differ or expand on what has been tried previously?
10%
Phase 2: Semifinalist Phase In this Phase, Participants must produce a set of proposed retrosynthesis pathways for each of the 10 or more Sponsor-provided Test Molecules, ranked by a parameter indicating likelihood of success (the Confidence Estimate). As described earlier, one or more of these Test Molecules will be presented in real time with Judges present to validate the time-effectiveness and autonomy of the platform solution. At the end of the Semifinalist Phase, Participants will be evaluated by a Judging Panel of 6 or more Judges. At least three Judges will have primary expertise in synthetic organic chemistry, and at least three will have primary expertise in computer-assisted synthesis and/or computer-assisted reaction prediction. The objective of the Judging Panel is to assess the Participants on the following criteria:
Criteria
Screening
Evaluation Type
Weight
Breadth of solution
For what percentage of the Test Molecules did the solution provide pathways that were deemed achievable?
Quantitative
25%
Chemical Intuition
Were the transformations in the solution novel and meaningfully different from reactions existing in the literature?
Semi- quantitative assessment by the Judges
25%
Confidence estimation
Did the solution provide a confidence score for each reaction step? Was this confidence score meaningful?
Semi- quantitative assessment by the Judges
15%
Parameter selection
Does the solution provide a mechanism to prioritize or deprioritize pathways based on parameters such as starting material, excluded reagents/solvents, or other parameters preferred by the user? If not, is there a proposal for how to incorporate this mechanism?
Quantitative
15%
Real-time evaluation
For the real-time demonstration, did the solution provide a plausible set of prioritized reaction pathways within a reasonable time period?
Quantitative
20%
Phase 3: Finalist Phase In this Phase, Participants must produce a set of proposed retrosynthesis pathways for each of the 5 or more Sponsor-provided Challenge Molecules, ranked by a parameter indicating likelihood of success (the Confidence Estimate). Each Participant will identify reaction steps that are produced by the chemical intuition capability of the solution (i.e., show that each such reaction step is not found in the public literature, but is plausible based on the public literature). Sponsor will choose one or more of these reaction steps to validate in the lab. At the end of the Finalist Phase, Participants will be evaluated by a Judging Panel of 6 or more Judges. At least three Judges will have primary expertise in synthetic organic chemistry, and at least three Judges will have primary expertise in computer-assisted synthesis and/or computer-assisted reaction prediction. The objective of the Judging Panel is to assess the Participants on the following criteria:
Criteria
Screening
Evaluation Type
Weight
Comprehensiveness and accuracy of solution
For what percentage of the challenge molecules did the solution provide pathways that were deemed achievable?
Did the platform provide accurate literature references for transformations?
Was there the appropriate amount of usable metadata? Was this metadata accurate?
Semi-quantitative assessment by the Judges
45%
Novelty
Can the solution identify when there is no good known route?
Does the solution then propose novel routes?
How accurate are these proposed novel routes?
Semi- quantitative assessment by the Judges
15%
Prioritization Mechanism and Confidence Estimation
Does the system provide a ranked list of pathway options?
Is there functionality that explains ranking results?
How well does the ranking algorithm perform in the current solution?
Does the parameterization evaluated in the Semi Finalist Phase perform well in the solution?
Semi- quantitative assessment by the Judges
40%
Award Purse
The total award purse for this challenge amounts to One Million Four Hundred Thousand U.S. Dollars ($1,400,000). Sponsor expects that these awards will not only motivate Participants to produce groundbreaking work, but also provide financial support to the awarded Participants in advancing their research and contributing to developments in chemical synthesis and AI/ML:
Grand Award: The Challenge Winner (Finalist with the AI/ML model that best fulfills the Challenge goal and evaluation Criteria) will be awarded One Million U.S. Dollars ($1,000,000), subject to the terms set out in these Guidelines. This substantial award highlights Sponsor’s dedication to encouraging innovations in the field of organic synthesis and AI/ML, and it underscores the transformative potential of the winning solution.
Finalist Award: Up to three (3) Semifinalists will be awarded One Hundred Thousand U.S. Dollars ($100,000) each,subject to the terms set out in these Guidelines.
White Paper Award: Up to ten (10) Participants with White Papers judged to best meet the Judging criteria for advancing to the Semifinalist Phase (Semifinalists) will be awarded Ten Thousand U.S. Dollars ($10,000) each, subject to the terms set out in these Guidelines.
To collect a White Paper Award or a Finalist Award, awarded Participants must continue to the next phase of the contest.
We are pleased to announce the successful completion of Phase 2 of the Standard Industries Chemical Innovation Challenge. We extend our heartfelt thanks to all the participant teams for their dedication and hard work throughout this challenging phase. Your innovative models are truly impressive and demonstrate exceptional creativity and potential.
The judging phase is scheduled to begin on September 16 and conclude on October 11. Our panel of experts is eager to review the detailed and pioneering solutions you have developed. Please note, these dates are subject to change if additional time is required to thoroughly evaluate the groundbreaking submissions we have received.
We are incredibly grateful for the commitment and expertise shown by each team, and we look forward to celebrating the remarkable achievements that have emerged from this contest.
Thank you once again for your contributions to advancing chemical synthesis through AI and ML technologies. Stay tuned for further updates!
One of the teams sent a file requesting a Scalability Index proposal for additional reaction types. The organizers promised to send our answers equally to every team. Each team can define its own measure of scalability. The scalability by reaction type list which has been added to the Team Shared Folders is meant to help and provide some guidance and feedback only.
Please refer to the scalability questions on HeroX for more guidance.
We are pleased to inform you that today we have sent the first batch of test molecules to the ten teams who have made it to the semi-finals. We are sending these one week ahead of the original schedule to give you more time to work on your solutions.
Please note that the time and date of the online demo remain as scheduled.
If you have any questions or need further assistance, please do not hesitate to contact us.
Good luck, and we look forward to seeing your innovative solutions!
Below please find answers to the questions asked by SI Chemistry Challenge Teams during our meetings in July. The questions are organized by type. If you have any additional questions, please do not hesitate to contact us.
Scalability
What happens if the solution has a different reaction type compared to what was asked?
Each team can define its own measure of scalability. The scalability by reaction type list provided by the organizers is now meant to help but it is not a complete list.
Do we want teams to provide a scalability index for the entire pathway, or just the scalability index for each step?
We would like teams to provide a scalability index for each step in the pathway
Can the organizers provide additional scoring for different reaction classes?
Any team can send us additional reaction types and we will get back to them. We will share that information with all teams. One team has already sent a long list and we expect to distribute our analysis next week.
Some reactions never appear in the list of reactions but we still get new and novel reactions. Is it positive or negative in our opinion?
For us the question is “does it work and is it scalable”. So we don’t mind if the Team’s submit reactions that don’t appear on the list.
What if a reaction with two or more different reaction types on the list?
One of the reasons that we’ve moved away from requiring the scalability by reaction list method is that it has imperfections like this. This list is provided as a guideline. One method that you could use is to average the scalability scores if the reaction falls in two classes.
Starting Materials
How should teams estimate the cost of input materials for their pathways?
Teams should focus on relative cost using the best data available from public domain sources, such as Sigma Aldrich or web based vendors using the prices relevant to the largest scale available
Are there any recommendations as to materials costs etc.
Cost and availability are used to determine the termination points of the tree search for backward reactions. “True” costs vary widely and there is no single source of truth, and in this stage, semifinalists can use whatever source they want. We suggest erring on the side of caution and go further back 1-3 steps if you are uncertain of the viability or cost of a starting material. This will not be penalized in the judging process. The organizers will likely have asymmetric information about the cost, and will pick which step to start the synthesis. An expanded JSON file will be distributed with the reactant metatag (Role: “StartingMaterial”) to identify the starting material you believe is viable.
Question about the database: do you have any restrictions in terms of the database we use for feedstock?
We will not prescribe the database that you use for starting point availability. The general rule is in section 4.1.6; use the biggest quantity that’s available. We will end up asking what database was used.
What are acceptable commercial sources of starting materials?
There is no single source of truth when it comes to pricing, especially given that cost is often a function of purity or reagent grade (ACS, USP, etc). Please use any source or grade, with the exception of "technical" or <95% purity. We may ask for your source over time.
Can we provide a list of green solvents?
We choose not to provide this because we don’t want to exclude reactions that are performed in non-green solvents, because a skilled organic chemist can often adapt the solvent system
Platform Runtime
Is runtime a critical factor?
This is a relatively low throughput exercise, so accuracy and rank are more important than time, but we want to make sure that the runtime can be done in the minutes to hours time frame rather than days. Runtime depends on the number of compute cycles and available processing power for the participant, so a direct comparison of time between the semifinalists will not be meaningful.
Question about the time judging.
Allowed run-time could be a parameter that your platform can control (e.g. give me your best answer within 30 min.)
Confidence Estimate
Does the confidence estimate measure the likelihood of success for the pathway based on observed results or based on literature support?
Each semifinalist can choose their own method of estimating the confidence estimate for each reaction step. This is intended to me a measure of how likely the reaction will work (e.g. will the reaction produce the desired products as the major products of the reaction step)?
Real-Time Demo
Will the real time demo on Aug be scored?
Yes, there will be a portion of real time demo scoring.
Will there be some time to present some interim choices?
The Aug. live call is the opportunity to discuss that.
For the realtime demo: is an hour the plan?
Yes
Final Stage Questions
In the final stage, will the PoC be done in a lab?
Our plan is to verify a few key steps in a laboratory environment.
Some of the advanced features which I’ve submitted will not be in the prototype, but only in the second phase.
Yes, no problem. This is aligned with our expectations.
Reaction Metrics Questions
How much should yield be taken into account in the ranking?
Yield is a very nice to have but not a required parameter since it is very difficult to estimate
What should be included in the detailed description of the reaction conditions
As with yields, reaction conditions are preferred but not required since they are not included in all training sets. They may help the judges estimate the likelihood of success and scalability of a reaction. Reaction conditions could include temperature, pressure, solvents.
JSON Questions
JSON - What can we add to the meta-data fields?
The semifinalists are free to add different kinds of meta-data for the reactants, products, reagents and the reaction step. Some useful attributes for molecules include name, boiling point, CAS#, unit cost. Some useful attributes for the reaction step include yield, temperature, pressure, pH. None of these are required to be included, but you can understand how they would help the judges estimate the likelihood of success of a reaction step.
What kind of units do we expect to see? (e.g. gr, mg, etc.)?
We are seeking SI units. Sometimes there is a choice between types (e.g. mols vs g). Either approach is acceptable. In an update of the JSON file we will include a method to transmit information about the custom meta data and what units are used.
JSON template vs real live presentation: in the template is there an opportunity to provide additional text?
In the template there’s a meta data tag, which isn’t mandatory but can be useful if you’d like to provide metadata for a given pathway.
Other Questions
Do you consider the set of rules as part of the system? The system will contain a model which would allow extracting templates, rules etc. So are the set of rules should be fixed or do we want the system to produce rules?
The system can be a mixture of AI-based and rule-based approaches or a hybrid approach where the AI derives templates for the rules.
Have you used other systems, some of them are free (mentioned several names)?
The organizers have tried many available approaches. In general, our observation is that they stay very close to their training data, and pay very little attention to scalability considerations in their prioritization of reaction pathways.
Legal Questions
What are the restrictions in terms of publishing or making the platform open for the public? What kinds of things can we publish?
Anything in the White Paper is safe to publish.
Our goal is for the Challenge Winner of the entire challenge to have an ongoing relationship with SI, and there are some legal terms for the Winner which could be jeopardized if the solution is fully published. The Winner of the prize is limited to “to not publicly disclose the Winning Solution’s software code, in particular in open source software form, for five (5) years”. Because the Challenge Winner is required to not publicly disclose the winning software code, it is an implied obligation that while in the competition, Participants can not publicly disclose or publish the code.
In the semifinalist phase, we suggest that the teams not publish their source code until the Finalists are selected (Oct 2024).
For a team that advances to the next phase (finals), we suggest they further wait until a Winner is chosen (Feb 2025) before publishing their source code.
We also limit all Participants via the NDA to not publish (i.e., keep confidential) SI Confidential Information, including the retrosynthesis pathways generated for the challenge.
Question about background vs. foreground IP
Advise you to consult with your own legal council.
Our platform uses LLMs and tools built on them. We can utilize both commercial LLMs (e.g. chatgpt) or locally hosted LLMs. We will test both to assess them. When using chatgpt we are likely to share context over the internet - is that problematic?
In the NDA we are protecting the fact that Grace and SI are working with molecules even if the molecules are public information. When using commercial LLMs like ChatGPT, that doesn’t disclose such information. We may label specific Test Molecules that should not use commercial LLMs but most will be able to use them.
Parameterization Controls and Prioritization Questions
Are there any particular controls SI is interested in, as part of the solution.
We would like to be able to prioritize or de-prioritize things like:
Reaction types: “do not use oxidation reactions “
Reaction conditions: “only include reaction temperatures greater than -40C and less than 150C”
Reagent costs: “do not use starting materials that cost more than $10/kg”
Reagent types: “do not use precious metal catalysts”
Confidence level: “include only reactions which have a >.8 confidence metric”
Scalability level: “include only reaction steps which have a scalability greater than 10”
We realize that these controls may not be ready in time for the real-time demonstration, but in that case the semifinalists and the judges could have a conversation about how or it such controls could be added.
The Intellectual Property described in the “Finalist Award” section of the OFFICIAL CHALLENGE RULES, TERMS & CONDITIONS excludes Participants’ AI/ML models and any changes thereto created during the Challenge. The Participants’ AI/ML models and any changes thereto created during the Challenge will be treated as Confidential Information of Participant consistent with the Non-Disclosure Agreement between the Sponsor and the Participant.
