Scala versus competition

C++ gives the programmer direct control of the machine and is fully compiled, meaning it will run exactly as intended. Scala runs on top of the JVM, using JIT (just-in-time) compilation. For maximum performance, both platforms need to be tuned by experts. C++ is more popular because of how long it has been used in the high-performance computing industry as well as game programming. Scala provides higher level programming capabilities that reduce the amount of code needed to achieve the same tasks, and more recent versions of C++ have taken inspiration from Scala. C++ is currently a better choice for general HPC and when you have at least a few external libraries to integrate with. However, if ease of understanding of code, and unit testing are crucial, especially for collaboration with business users, Scala is a solid choice.

Kotlin is a more light-weight language, made to make the JVM programmer's life easier. Scala's aim is entirely different: combine advanced functional programming and object-oriented programming on the JVM, and create a scalable language.

While Kotlin is not as powerful as Scala, the learning curve is much smaller especially for those coming from Java, because it has fewer functional programming principles and tries to look as familiar as possible; in 2019, it was anointed with the title of Android's primary programming language. In terms of skills demand, its growth is high but Scala still edges it out in total numbers (check on Indeed.com).

However, the reason for Kotlin's ease of learning is simple: it lacks a meta-programming system and advanced type system like Scala's, as well as syntactical flexibility. Meta-programming enables you to generate mappings, CSV encoders, JSON codecs, and manipulate code to create new code,  thereby reducing repetition and possible run-time errors; you can generate HTML forms from classes, and write injection-free SQL queries (the 'doobie' library), but at the investment of a steeper learning curve.

For data science and machine learning, Scala is the perfect choice because of libraries like Apache Spark that are written in Scala; if you do plan to extend your back-end application with data science, then it should also be written in Scala. Kotlin should be chosen where you want a better Java without a significant time investment, or for Android - however do note that more recently, Oracle has started to get its game together with the introduction of record types and pattern matching in the JDK.

Scala compiles down to Java/JVM code/bytecode and based itself on the Java platform. Developing in Scala is faster for the key reasons of more concise syntax, advanced type-checking (there are libraries like doobie which prevents SQL injections for database access and do type checking). Java is fast catching up in the recent years, in particular with support for Optionals, Streams, Lambdas and Record types, though it looks more like catching up with Kotlin than Scala.

Pure-functional programming is far more difficult in Java because it was not designed with it in mind. However, for frameworks like Spring, JavaEE and Dropwizard, especially from skills and resourcing perspective, Java is the best choice because of the sheer amount of documentation out there, and even better than Kotlin from that aspect as well.

If you are new to the JVM, definitely start with Scala, and if you are not, definitely try it out. If you have an existing Java project, enhance it with ScalaTest, but make sure that the whole team is moving at a similar pace. In Maven, Gradle and SBT, mixed-language projects of Scala and Java are no challenge, and changing from Java to Java-like Scala is trivial. The JetBrains IDE IntelliJ, converts Java automatically to Scala when pasting.

Go has a great advantage of being a small, simple, and easy to use language. It comes with batteries installed and development by convention is straightforward. Go is used in popular system and DevOps projects like Kubernetes, OpenShift and Hashicorp Nomad, and it will most likely remain in this position because it integrates well with UNIX systems and binaries are very simple to distribute. Go, Rust and C/C++ remain top contenders for command-line and native applications in the current day and age.

Go is also popular for REST APIs and backend applications, however Scala would be a better choice for large back-end applications because of its support for Generics, very advanced type-safety, property testing libraries as well as highly scalable syntax and re-use of the huge set of JVM's libraries and JMX ("Java Management Extensions"), which enable you to manage your application remotely through very powerful JVM tooling, without having to create your own DIY solutions. Companies like Uber have very large systems in Go, however they had to spend effort to build bespoke tooling to handle this.

It does not have to be a tug of war - in fact, one of the largest projects out there, Apache Spark, is built in Scala but with R and Python wrappers. As Python gets older, it imports more features from Scala like type-safety, but Scala was build ground-up for that. Python has an easy learning curve and the newcomer documentation and resources are excellent. Performance in Scala is much higher because it uses the JVM and is compiled, whereas Python is interpreted, although there are ways to speed it up. Python has a much richer set of libraries for data scientists. Recommended is to use Python for speed of prototyping but then productionize the application in Scala.

For REST APIs and back-ends, Python is better for smaller applications, however difficulties come when more advanced scalability and monitoring features are required: Scala application development does not slow down with a larger codebase unlike dynamically typed languages; and Scala fully leverages the JVM platform which provides JMX ("Java Management Extensions") enabling far more standardised remote management of your applications among other incredibly powerful tooling.

Scala is heavily inspired by Haskell with regards to its pure-functional programming concepts. In fact, in the 1990s, Martin Odersky and Philip Wadler worked together on the Generic Java compiler which made its way to Java as "Generics", revolutionising Java.

Haskell is used in the industry, however it has been challenging to integrate it with enterprise ecosystems that were build on top of the JVM platform, and also because it constantly changing as research language. For Scala, it is much easier because it interoperates with Java directly without any extra layers, meaning that the speed of building on top of the existing platform is much easier. Learning Haskell is hard because its syntax is even less familiar than that of Scala, however once learned, it can be even more concise. Anecdotal evidence says that Scala developers secretly wish they were doing Haskell.

Haskell loses its advantages in terms of system integrations and is best suited for more self-contained applications that have fewer external dependencies: the more integrations with the external world, the more the 'IO boundary' needs to be breached, the more boilerplate the code becomes and starts to look less like Haskell.

While the question of Scala vs Rust is posed very often, in fact Rust competes mainly with C++ in the space of command line programs, kernel programming and the Firefox's browser engine, and other cases where high performance is needed, but so is reliability. With Rust you lose the JVM capabilities like JMX, but you also gain in small programs with very fast start-up time. In native integration or instant start-up are needed for a Scala programmer, they should try Oracle's GraalVM, scala-native or Java Native Interface as well as Java's direct memory access, before going with Rust because a lot will be missed from the JVM; on the other hand somebody already familiar with Rust would benefit from Scala the most if going into Big Data & data science (although Python might make sense as well).

Because of this, the comparison is not so straightforward and depends on your situation.

TypeScript, a super-set of JavaScript, introduced a type checker and compiler on top of JavaScript, enabling more scalable development especially from the perspective of self-documenting components and their re-use. TypeScript and JavaScript run both on all web browsers and on the server side for back-end application with Node.js.

For the back-end, if you plan to grow your application a lot and also take on data processing, Scala should be an easy choice especially because it runs on the JVM which provides JMX ("Java Management Extensions") which enable out-of-bound management of your application at runtime (no need to bloat your API with management and metrics endpoints). Scala.js can also run on Node.js on the back-end without issue if needed as well; here, the JVM advantage is lost, but then the NPM ecosystem can be leveraged.

For the front-end, Scala also targets browsers via Scala.js. The vast majority of front-end documentation is in JavaScript meaning fairly comprehensive JavaScript knowledge is still required; there are very few Scala.js developers and many of them come from back-end background so they are not so familiar with the intricacies of HTML5 APIs, CSS and visual design. The learning curve much steeper than that of TypeScript, but there is one case where using Scala on the full stack makes a lot of sense: where you are performing the same computations on the back-end and front-end and you need to be 100% certain that contractual numbers are represented correctly and consistently, especially when creating contracts with your customers in trading systems.

Scala's biggest advantages are that it is built on top of the JVM and has very flexible syntax that is tamed by its advanced type verification ("strong typing") system enabling highly scalable development in the long run, especially for much larger and complex applications requiring precision and certainty.

There are two ways to do production Scala: replacing unit tests with Scala in an existing Java codebase; otherwise starting a new project. Generally, migrating from any language to any other language is challenging.

Currently, the two most popular use cases are back-end APIs and data science projects, which is perfect companies in the finance industry and the likes of Twitter, Netflix, eBay, LinkedIn among many others, processing vast amounts of data on-prem and in the cloud. For other use cases, consider the most appropriate tool but also feel free to reach out if you are looking for advice.

Back-end APIs are most commonly powered by akka-http, http4s and Play frameworks. Data science is typically done with Apache Spark, however is also done in pure Scala.