So long, and thanks for only using readily available scripts

Purpose It is often argued that the increased automation and availability of offensive cyber tools has decreased the skill and knowledge required by attackers. Some say that all it takes to succeed with an attack is to follow some instructions and push some buttons. This paper aims to tests this idea empirically through live exploits and vulnerable machines in a cyber range. Design/methodology/approach The experiment involved 204 vulnerable machines in a cyber range. Exploits were chosen based on the results of automated vulnerability scanning. Each exploit was executed following a set of carefully planned actions that enabled reliable tests. A total of 1,223 exploitation attempts were performed. Findings A mere eight exploitation attempts succeeded. All these involved the same exploit module (ms08_067_netapi). It is concluded that server-side attacks still are too complicated for novices who lack the skill or knowledge to tune their attacks. Originality/value This paper presents the largest conducted test of exploit effectiveness to date. It also presents a sound method for reliable tests of exploit effectiveness (or system vulnerability).