We compare the absolute and relative performance of three approaches to predicting outcomes for entrants in a business plan competition in Nigeria: Business plan scores from judges, simple ad hoc prediction models used by researchers, and machine learning approaches. We find that i) business plan scores from judges are uncorrelated with business survival, employment, sales, or profits three years later
ii) a few key characteristics of entrepreneurs such as gender, age, ability, and business sector do have some predictive power for future outcomes
iii) modern machine learning methods do not offer noticeable improvements
iv) the overall predictive power of all approaches is very low, highlighting the fundamental difficulty of picking competition winners.