Presentation Scala: The Simple Parts (Keynote)

I'd like to take you on a journey to what I think of as the core of Scala. The core is built from a moderate number of general and orthogonal concepts that can be combined freely. The parts are simple, but the combinations can be as elaborate s one wants to make them.


When done right, simple parts in the language lead to libraries that have clear interfaces and can be used in flexible ways. I will give examples how the fusion of functional and object-oriented concepts in Scala helps in the design of simple modules that compose well.

Speakers


PDF: slides.pdf

Slides

Welcome!

Welcome! Scala Days 2014 @scaladays

Topics

Topics • • • • • • House keeping items Scala Days App Demo Session voting Phil Bagwell Award Keynote: Martin Odersky World Cup and BBQ

Welcome!

Welcome! Scala Days 2014 @scaladays

Topics

Topics • • • • • • House keeping items Scala Days App Demo Session voting Phil Bagwell Award Keynote: Martin Odersky World Cup and BBQ

Scala Days 2014

Scala Days 2014 Android & iPhone

Seattle

Seattle Spain Creative Consulting

@47deg

@47deg http://47deg.com Thanks & Enjoy!

In memory of Phil Bagwell

In memory of Phil Bagwell

Lalit Pant, founder and creator of Kojo

Lalit Pant, founder and creator of Kojo

Lalit Pant, founder and creator of Kojo

Lalit Pant, founder and creator of Kojo

Scala - The Simple Parts

Scala - The Simple Parts Martin Odersky Typesafe and EPFL

10 Years of Scala

10 Years of Scala

Grown Up?

Grown Up? Scala’s user community is pretty large for its age group. ~ 100’000 developers ~ 200’000 subscribers to Coursera online courses. #13 in RedMonk Language Rankings Many successful rollouts and happy users. But Scala is also discussed more controversially than usual for a language at its stage of adoption. Why?

Controversies

Controversies Internal controversies: Different communities don’t agree what programming in Scala should be. External complaints: “Scala is too academic” “Scala has sold out to industry” “Scala’s types are too hard” “Scala’s types are not strict enough” “Scala is everything and the kitchen sink” Signs that we have not made clear enough what the essence of programming in Scala is.

Object-Oriented

Object-Oriented The Picture So Far Agile, with lightweight syntax Functional = scalable Safe and performant, with strong static tpying 1-

What is “Scalable”?

What is “Scalable”? 1st meaning: “Growable” – can be molded into new languages by adding libraries (domain specific or general) See: “Growing a language” (Guy Steele, 1998) 2nd meaning: “Enabling Growth” – can be used for small as well as large systems – allows for smooth growth from small to large.

A Growable Language

A Growable Language • Flexible Syntax • Flexible Types • User-definable operators • Higher-order functions • Implicits ... Make it relatively easy to build new DSLs on top of Scala And where this fails, we can always use the macro system (even though so far it’s labeled experimental)

A Growable Language

A Growable Language SBT Chisel Spark Spray Dispatch shapeless Akka Scalaz ScalaTest Specs Slick Squeryl

Growable = Good?

Growable = Good? In fact, it’s a double edged sword. – DSLs can fracture the user community (“The Lisp curse”) – Besides, no language is liked by everyone, no matter whether its a DSL or general purpose. ��� Host languages get the blame for the DSLs they embed. Growable is great for experimentation. But it demands conformity and discipline for large scale production use.

A Language For Growth

A Language For Growth • • • • Can start with a one-liner. Can experiment quickly. Can grow without fearing to fall off the cliff. Scala deployments now go into the millions of lines of code. – Language works for very large programs – Tools are challenged (build times!) but are catching up. “A large system is one where you do not know that some of its components even exist”

What Enables Growth?

What Enables Growth? Unique combination of Object/Oriented and Functional Large systems rely on both. Object-Oriented Functional Unfortunately, there’s no established term for this object/functional?

Scala’s Role in History ☺

Scala’s Role in History ☺ (from:James Iry: A Brief, Incomplete, and Mostly Wrong History of Programming Languages)

Another View: A Modular Language

Another View: A Modular Language Large Systems Object-Oriented Functional = modular Small Scripts

Modular Programming

Modular Programming Systems should be composed from modules. Modules should be simple parts that can be combined in many ways to give interesting results. – (Simple: encapsulates one functionality) But that’s old hat! – Should we go back to Modula-2? – Modula-2 was limited by the Von-Neumann Bottleneck (see John Backus’ Turing award lecture). – Today’s systems need richer models and implementations.

FP is Essential for Modular Programming

FP is Essential for Modular Programming Read: “Why Functional Programming Matters” (John Hughes, 1985). Paraphrasing: “Functional Programming is good because it leads to modules that can be combined freely.”

Functions and Modules

Functions and Modules Functional does not always imply Modular. Some concepts in functional languages are at odds with modularity: – Haskell’s type classes – OCaml’s records and variants – Dynamic typing? (can argue about this one)

Objects and Modules

Objects and Modules Object-oriented languages are in a sense the successors of classical modular languages. But Object-Oriented does not always imply Modular either. Non-modular concepts in OOP languages: – Smalltalk’s virtual image. – – – – Monkey-patching Mutable state makes transformations hard. Weak composition facilities require external DI frameworks. Weak decomposition facilities encourage mixing domain models with their applications.

Scala – The Simple Parts

Scala – The Simple Parts Before discussing library modules, let’s start with the simple parts in the language itself. They correspond directly to the fundamental actions that together cover the essence of programming in Scala compose – match – group – recurse – abstract – aggregate – mutate As always: Simple ≠ Easy !

1. Compose

1. Compose Everything is an expression → everything can be composed with everything else. if (age >= 18) "grownup" else "minor" val result = tag match { case “email” => try getEmail() catch handleIOException case “postal” => scanLetter() }

2. Match

2. Match Pattern matching decomposes data. It’s the dual of composition. trait Expr case class Number(n: Int) extends Expr case class Plus(l: Expr, r: Expr) extends Expr def eval(e: Expr): Int = e match { case Number(n) => n case Plus(l, r) => eval(l) + eval(r) } Simple & flexible, even if a bit verbose.

The traditional OO alternative

The traditional OO alternative trait Expr { def eval: Int } case class Number(n: Int) extends Expr { def eval = n } case class Plus(l: Expr, r: Expr) extends Expr { def eval = l.eval + r.eval } OK if operations are fixed and few. But mixes data model with “business” logic.

2. Match

2. Match Pattern matching decomposes data. It’s the dual of composition. trait Expr case class Number(n: Int) extends Expr case class Plus(l: Expr, r: Expr) extends Expr def eval(e: Expr): Int = e match { case Number(n) => n case Plus(l, r) => eval(l) + eval(r) } Simple & flexible, even if a bit verbose.

The traditional OO alternative

The traditional OO alternative trait Expr { def eval: Int } case class Number(n: Int) extends Expr { def eval = n } case class Plus(l: Expr, r: Expr) extends Expr { def eval = l.eval + r.eval } OK if operations are fixed and few. But mixes data model with “business” logic.

3. Group

3. Group Everything can be grouped and nested. Static scoping discipline. Two name spaces: Terms and Types. Same rules for each. def solutions(target: Int): Stream[Path] = { def isSolution(path: Path) = path.endState.contains(target) allPaths.filter(isSolution) }

Tip: Don’t pack too much in one expression

Tip: Don’t pack too much in one expression I sometimes see stuff like this: jp.getRawClasspath.filter( _.getEntryKind == IClasspathEntry.CPE_SOURCE). iterator.flatMap(entry => flatten(ResourcesPlugin.getWorkspace. getRoot.findMember(entry.getPath))) It’s amazing what you can get done in a single statement. But that does not mean you have to do it.

Tip: Find meaningful names!

Tip: Find meaningful names! There’s a lot of value in meaningful names. Easy to add them using inline vals and defs. val sources = jp.getRawClasspath.filter( _.getEntryKind == IClasspathEntry.CPE_SOURCE) def workspaceRoot = ResourcesPlugin.getWorkspace.getRoot def filesOfEntry(entry: Set[File]) = flatten(workspaceRoot.findMember(entry.getPath) sources.iterator flatMap filesOfEntry

4. Recurse

4. Recurse Recursion let’s us compose to arbitrary depths. This is almost always better than a loop. Tail-recursive functions are guaranteed to be efficient. @tailrec def sameLength(xs: List[T], ys: List[U]): Boolean = if (xs.isEmpty) ys.isEmpty else ys.nonEmpty && sameLength(xs.tail, ys.tail)

5. Abstract

5. Abstract Functions are abstracted expressions. Functions are themselves values. Can be named or anonymous. def isMinor(p: Person) = p.age < 18 val (minors, adults) = people.partition(isMinor) val infants = minors.filter(_.age <= 3) (this one is pretty standard by now) (even though scope rules keep getting messed up sometimes)

6. Aggregate

6. Aggregate Collection aggregate data. Transform instead of CRUD. Uniform set of operations Very simple to use Learn one – apply everywhere

Collection Objection

Collection Objection “The type of map is ugly / a lie”

Collection Objection

Collection Objection “The type of map is ugly / a lie”

Why CanBuildFrom?

Why CanBuildFrom? • Why not define it like this? class Functor[F[_]] { def map[T, U](f: T => U): F[T] => F[U] } • Does not work for arrays, since we need a class-tag to build a new array. • More generally, does not work in any case where we need some additional information to build a new collection, (e.g. OrderedSet, String). • This is precisely what’s achieved by CanBuildFrom.

Collection Objection

Collection Objection “The type of map is ugly / a lie”

Collection Objection

Collection Objection “The type of map is ugly / a lie”

Why CanBuildFrom?

Why CanBuildFrom? • Why not define it like this? class Functor[F[_]] { def map[T, U](f: T => U): F[T] => F[U] } • Does not work for arrays, since we need a class-tag to build a new array. • More generally, does not work in any case where we need some additional information to build a new collection, (e.g. OrderedSet, String). • This is precisely what’s achieved by CanBuildFrom.

7. Mutate

7. Mutate It’s the last point on the list and should be used with restraint. But are vars and mutation not anti-modular? – Indeed, global state often leads to hidden dependencies. – But used-wisely, mutable state can cut down on boilerplate and increase efficiency and clarity.

Where I use Mutable State

Where I use Mutable State In dotc, a newly developed compiler for Scala: – caching – – – – lazy vals, memoized functions, interned names, LRU caches. persisting once a value is stable, store it in an object. copy on write avoid copying untpd.Tree to tpd.Tree. fresh values fresh names, unique ids typer state 2 vars: current constraint & current diagnostics (versioned, explorable).

Why Not Use a Monad?

Why Not Use a Monad? The fundamentalist functional approach would mandate that typer state is represented as a monad. Instead of now: def typed(tree: untpd.Tree, expected: Type): tpd.Tree def isSubType(tp1: Type, tp2: Type): Boolean we’d write: def typed(tree: untpd.Tree, expected: Type): TyperState[tpd. Tree] def isSubType(tp1: Type, tp2: Type): TyperState [Boolean]

Why Not Use a Monad?

Why Not Use a Monad? Instead of now: if (isSubType(t1, t2) && isSubType(t2, t3)) result we’d write: for { c1 A Question of Typing A Question of Typing Clojure Scala syntax arguments Haskell effects Idris Coq values correctness Statically checked properties None of the 5 languages above is “right”. It’s all a question of tradeoffs.

Why Not Use a Monad?

Why Not Use a Monad? The fundamentalist functional approach would mandate that typer state is represented as a monad. Instead of now: def typed(tree: untpd.Tree, expected: Type): tpd.Tree def isSubType(tp1: Type, tp2: Type): Boolean we’d write: def typed(tree: untpd.Tree, expected: Type): TyperState[tpd. Tree] def isSubType(tp1: Type, tp2: Type): TyperState [Boolean]

Why Not Use a Monad?

Why Not Use a Monad? Instead of now: if (isSubType(t1, t2) && isSubType(t2, t3)) result we’d write: for { c1 Why Not Use a Monad? Why Not Use a Monad? The fundamentalist functional approach would mandate that typer state is represented as a monad. Instead of now: def typed(tree: untpd.Tree, expected: Type): tpd.Tree def isSubType(tp1: Type, tp2: Type): Boolean we’d write: def typed(tree: untpd.Tree, expected: Type): TyperState[tpd. Tree] def isSubType(tp1: Type, tp2: Type): TyperState [Boolean]

Where I use Mutable State

Where I use Mutable State In dotc, a newly developed compiler for Scala: – caching – – – – lazy vals, memoized functions, interned names, LRU caches. persisting once a value is stable, store it in an object. copy on write avoid copying untpd.Tree to tpd.Tree. fresh values fresh names, unique ids typer state 2 vars: current constraint & current diagnostics (versioned, explorable).

Why Not Use a Monad?

Why Not Use a Monad? Instead of now: if (isSubType(t1, t2) && isSubType(t2, t3)) result we’d write: for { c1 A Question of Typing A Question of Typing Clojure Scala syntax arguments Haskell effects Idris Coq values correctness Statically checked properties None of the 5 languages above is “right”. It’s all a question of tradeoffs.

From Fundamental Actions to Simple Parts

From Fundamental Actions to Simple Parts Compose Expressions Mutate Nest Scopes Vars Aggregate Match Collections Abstract Function Values Recurse Functions Patterns

Modules

Modules Modules can take a large number of forms – A function – – – – – An object A class An actor A stream transform A microservice Modular programming is putting the focus on how modules can be combined, not so much what they do. In Scala, modules talk about values as well as types.

Features For Modular Programming

Features For Modular Programming 1. Our Vocabulary: Rich types with static checking and functional semantics – gives us the domains of discourse, – gives us the means to guarantee encapsulation, – see: “On the Criteria for Decomposing Systems into Modules” (David Parnas, 1972). 2. 3. Start with Objects − atomic modules Parameterize with Classes − templates to create modules dynamically 4. Mix it up with Traits − mixable slices of behavior

5. Abstract By Name

5. Abstract By Name Members of a class or trait can be concrete or abstract. Example: A Graph Library

Where To Use Abstraction?

Where To Use Abstraction? Simple rule: – Define what you know, leave abstract what you don’t. – Works universally for values, methods, and types.

5. Abstract By Name

5. Abstract By Name Members of a class or trait can be concrete or abstract. Example: A Graph Library

Where To Use Abstraction?

Where To Use Abstraction? Simple rule: – Define what you know, leave abstract what you don’t. – Works universally for values, methods, and types.

Encapsulation = Parameterization

Encapsulation = Parameterization Two sides of the coin: 1. Hide an implementation 2. Parameterize an abstraction

5. Abstract By Name

5. Abstract By Name Members of a class or trait can be concrete or abstract. Example: A Graph Library

Encapsulation = Parameterization

Encapsulation = Parameterization Two sides of the coin: 1. Hide an implementation 2. Parameterize an abstraction

6. Abstract By Position

6. Abstract By Position Parameterize classes and traits. class List[+T] class Set[T] class Function1[-T, +R] List[Number] Set[String] Function1[String, Int] Variance expressed by +/- annotations A good way to explain variance is by mapping to abstract types.

Modelling Parameterized Types

Modelling Parameterized Types class Set[T] { ... } → class Set { type $T } Set[String] → Set { type $T = String } class List[+T] { ... } → class List { type $T }List [Number] → List { type $T <: Number } Parameters → Abstract members Arguments → Refinements

7. Keep Boilerplate Implicit

7. Keep Boilerplate Implicit Implicit parameters are a rather simple concept But they are surprisingly versatile! Can represent a typeclass: def min(x: A, b: A)(implicit cmp: Ordering[A]): A

Implicit Parameters

Implicit Parameters Can represent a context: def typed(tree: untpd.Tree, expected: Type) (implicit ctx: Context): Type def compile(cmdLine: String) (implicit defaultOptions: List[String]): Unit Can represent a capability: def accessProfile(id: CustomerId) (implicit admin: AdminRights): Info

Module Parts

Module Parts Rich Static Types The vocabulary Implicit Parameters Objects atomic modules cut boilerplate Type Parameters Classes parameterized modules abstract by position Abstract Types Traits slices of behavior abstract by name

Summary

Summary A fairly modest (some might say: boring) set of parts that can be combined in flexible ways. Caveat: This is my selection, not everyone needs to agree.

Other Parts

Other Parts Here are parts that are either not as simple or do not work as seamlessly with the core: – – – – – Implicit conversions Existential types Structural types Higher-kinded types Macros All of them are under language feature flags or experimental flags. – This makes it clear they are outside the core. – My advice: Use only with specific justification – e.g, you use Scala as a host for a DSL or other language.

Thank You

Thank You Follow us on twitter: @typesafe