The article is dedicated to the problem of comparing performance and usability of native and Java-based server-side scripting. A new benchmark was developed on the base of a real usage scenario of the dynamic Wiki page generation. It allows a direct performance comparison between web servers with different script engines such as PHP, Python, Ruby and the corresponding Java implementations. The measurements have shown that Java script engines may substantially outperform their native counterparts, while native scripting technologies are by far more developed and mature. On the other hand, all Java script engines had to be seriously modified in order to run the benchmark due to bugs and unimplemented features. Besides, JSR223 introduced in Java 6 to simplify script usage appeared to be weaker than the native CGI interface and therefore of limited applicability for server-side scripting, because of the API limitations and incomplete support from the script engines.