Cookbook
Optics are probably the most powerful unused tool in our arsenal — this page is the working proof. Runnable patterns for the questions that come up most often, organised by the three jobs optics do best:
- Navigating structures — reach one
branch of a sum, every node of a recursive tree, several fields
at once, or a field inside bytes you never decode. One
.andThensurface for all of it. - Decoupling and modularity — replace a direct dependence on a data type with an optic: depend on the relationship, ask for the weakest capability you need, and let independent modules interoperate at the call site.
- Effect threading — turn control flow into type definitions: the optic pins where, the effect type decides what happens — abort, accumulate, audit, defer.
The framing follows Hansen — We Need More Optics (https://medium.com/swlh/we-need-more-optics-8ddf1d2d9468) and Penner — Optics By Example (https://leanpub.com/optics-by-example/). These are worked examples, not a tutorial — for the optic families themselves see the Optics reference. Every fence is compiled by mdoc against the current library version, and every recipe cites its source.
If you arrive with a task rather than an optic in mind, start here:
| I want to… | Recipe |
|---|---|
| edit one branch of a sum type, misses passing through | Visit contingent fields |
| apply one edit at every node of a recursive tree | Visit across whole trees |
| update only the matching elements of a collection | Visit through arbitrary structure |
| fold several fields into one number | Isolate what you need |
| rewrite every slot of a fixed shape, or summarise a batch | Compute aggregations |
| update an off-diagonal grid coordinate or preserve row labels | Indexed: Grates and Glasses |
| change a field deep in JSON without decoding the payload | Edit JSON without decoding |
| depend on a relationship instead of a data type | Require the optic, not the type |
| ask for the weakest capability a function needs | Depend only on what's needed |
| edit Avro bytes on a hot path, no decode | Serdes free pipes |
| move one field between wire formats (Avro ↔ JSON) | Inspectable cross-format bridges |
| law-test the optics my module publishes | Re-usable laws for testing |
| make an in-place update fallible | Validate-in-place with modifyF |
| swap abort / accumulate / audit without touching the optic | Structure orthogonal to effects |
| batch N+1 lookups through one traversal | Batch-load nested IDs |
| stamp an effect's result back into the entity | Persist-and-stamp decode free |
import dev.constructive.eo.optics.{Lens, Optic, Prism, Traversal}
import dev.constructive.eo.optics.Optic.*
import dev.constructive.eo.data.MultiFocus.given // Functor / Foldable / Traverse for MultiFocus[PSVec] (post-fold)
Navigating structures
One branch of a sum; every node of a recursive tree; a sparse walk
over a collection; several fields at once; and the same moves over
JSON you never decode. The recipes ramp by shape, and the point
throughout is that no step dead-ends: whatever you just focused,
the next .andThen keeps going.
Visit contingent fields
Why: you want to rewrite one variant of a sum type — the bound
variable of a Var node in an AST, say — and leave every other
case untouched, without a hand-written match that re-builds the
misses. Both hops derive from the type itself: prism[S, A]
matches the branch, lens[S](_.field) reaches inside it, and one
.andThen chains them — no pattern match, no copy plumbing, no
optic written by hand.
// The AST under edit — hosted at package level in
// dev.constructive.eo.docs (macro-derived optics need top-level
// targets; mdoc wraps every fence in an object):
//
// enum Expr:
// case Var(name: String)
// case App(f: Expr, x: Expr)
// case Lam(bind: String, body: Expr)
import dev.constructive.eo.docs.Expr
import dev.constructive.eo.generics.{lens, prism}
val varName = prism[Expr, Expr.Var].andThen(lens[Expr.Var](_.name))
// The focus arrives as the case's `(name: String)` NamedTuple —
// the selector covers Var's only field, so the derived optic is a
// full BijectionIso rather than a partial Lens.
val upperVar = varName.modify(v => (name = v.name.toUpperCase))
// Hit branch: the Var's name is uppercased.
upperVar(Expr.Var("x"))
// res0: Expr = Var("X")
// Miss branch: passes through, the Lam is untouched.
upperVar(Expr.Lam("y", Expr.Var("y")))
// res1: Expr = Lam(bind = "y", body = Var("y"))
The derived Prism rides the Either carrier and the derived field
optic composes across the seam through the same .andThen as
everything else on this page. When you need a Prism the macro can't
express — a validating match, a normalising re-build — the
hand-rolled Prism[S, A](sel, emb) constructor takes the match
explicitly; see Generics → prism[S, A] for what
the derivation covers (enums, sealed traits, union types) and the
full story on the NamedTuple focus.
Source: Baeldung — Monocle Optics, https://www.baeldung.com/scala/monocle-optics; framing from Wlaschin — Domain Modeling Made Functional, ch. 4, https://pragprog.com/titles/swdddf/domain-modeling-made-functional/.
Visit across whole trees
varName edits exactly the node you hand it: Var("x") hits,
Lam(...) misses — the nested Var("y") inside stays untouched,
because a Prism looks at the top of the value, not into it. One
more derivation closes the gap: plate[Expr] derives the Plated
instance (which fields recurse, which are leaves), and
everywhere turns it into a recursive optic that reaches every
sub-term. The same varName optic then composes into it:
import dev.constructive.eo.optics.Plated
import dev.constructive.eo.generics.plate
given Plated[Expr] = plate[Expr]
// `everywhere` is `transform` in optic form: everywhere.andThen(d).modify(g)
// applies `d` at every node, bottom-up. Reuse `varName` from above.
val everyVarName = Plated.everywhere[Expr].andThen(varName)
val upperEveryVar = everyVarName.modify(v => (name = v.name.toUpperCase))
val term = Expr.App(Expr.Var("f"), Expr.Lam("y", Expr.Var("y")))
// term: Expr = App(f = Var("f"), x = Lam(bind = "y", body = Var("y")))
// varName alone: the top of `term` is an App, not a Var — a miss,
// nothing changes.
upperVar(term)
// res2: Expr = App(f = Var("f"), x = Lam(bind = "y", body = Var("y")))
// everywhere ∘ varName: every variable at every depth — f -> F and
// the nested y -> Y. The Lam binder "y" (a String, not a Var node)
// is left alone.
upperEveryVar(term)
// res3: Expr = App(f = Var("F"), x = Lam(bind = "y", body = Var("Y")))
That pair of calls is the whole comparison: varName is the
surgical single-node edit, everyVarName the same edit quantified
over the tree — and the step between them is one derivation plus
one .andThen, not a rewrite. The .modify runs at every node,
bottom-up and stack-safe to any depth (a million-node tree, or a
100k-deep degenerate spine, won't overflow). When the rewrite is easier
as a plain per-node function, Plated.transform(f) is the same engine;
Plated.universe / children are the read side (every sub-term /
immediate children); rewrite repeats an Expr => Option[Expr] rule
to a fixpoint. The derivation follows the exact self-type rule —
only Expr-typed fields are recursion points, so Lam's
bind: String stays a leaf. See Generics → plate[S].
Source: Mitchell & Runciman — Uniform Boilerplate and List
Processing (Uniplate),
https://ndmitchell.com/downloads/paper-uniform_boilerplate_and_list_processing-30_sep_2007.pdf;
Haskell lens Control.Lens.Plated,
https://hackage.haskell.org/package/lens/docs/Control-Lens-Plated.html.
Visit through arbitrary structure
Why: you have a list of results, some Ok and some Err, and
want to bump only the successes — leaving the failures exactly
where they are. Walking the container and matching the branch in
one pass is the "sparse traversal" that's genuinely annoying to
hand-roll; here it's a one-liner.
import scala.collection.immutable.ArraySeq
// Result (Ok | Err) is hosted in dev.constructive.eo.docs like Expr
// above — enum macro targets need a package-level home under mdoc.
import dev.constructive.eo.docs.Result
val okP = prism[Result, Result.Ok]
val bumpOks =
Traversal.each[ArraySeq, Result]
.andThen(okP)
// Result.Ok is single-field, where the full-cover lens macro returns
// an Iso with a NamedTuple focus — hand-write the Int lens instead.
.andThen(Lens[Result.Ok, Int](_.value, (o, v) => o.copy(value = v)))
bumpOks.modify(_ + 1)(
ArraySeq(Result.Ok(10), Result.Err("nope"), Result.Ok(20))
)
// res4: ArraySeq[Result] = ArraySeq(Ok(11), Err("nope"), Ok(21))
Performance: the PowerSeriesPrismBench suite measures this
shape at ~5× over a hand-rolled naive loop — and that's the
published worst case for each, not the typical one. The
per-benchmark curve lives in
benchmarks → PowerSeries with Prism inner.
Source: Penner — Optics By Example ch. 8 (Traversal Actions) + ch. 10 (Missing Values), https://leanpub.com/optics-by-example/.
Isolate what you need
Why: you have a basket of line items and want one number —
total value — without writing a fold by hand or threading two
fields through a loop. The lens[S](_.a, _.b) macro focuses both
quantity and price as a single Scala 3 NamedTuple, so the
multiply-and-sum drops straight into foldMap.
import cats.instances.list.given // Traverse[List] for .each
import cats.instances.double.given // Monoid[Double] for .foldMap
import dev.constructive.eo.generics.lens
case class OrderItem(sku: String, quantity: Int, price: Double)
// Focus both pricing fields at once, then walk every item.
val lineValue =
Traversal.each[List, OrderItem]
.andThen(lens[OrderItem](_.quantity, _.price))
val inventory = List(
OrderItem("apple", 3, 9.99),
OrderItem("pear", 10, 2.50),
OrderItem("lobster", 2, 45.00),
)
// inventory: List[OrderItem] = List(
// OrderItem(sku = "apple", quantity = 3, price = 9.99),
// OrderItem(sku = "pear", quantity = 10, price = 2.5),
// OrderItem(sku = "lobster", quantity = 2, price = 45.0)
// )
// One pass: see each item's (quantity, price) NamedTuple, multiply,
// and let the Double monoid sum the line values into a grand total.
lineValue.foldMap(nt => nt.quantity * nt.price)(inventory)
// res5: Double = 144.97
The focus arrives in selector order, so nt.quantity and
nt.price are named — the lambda reads like the business rule it
encodes. The same lineValue optic also writes (.modify to
re-price every line); the fold is just the read-side escape. See
Generics → Multi-field Lens for the full treatment
of the NamedTuple focus.
Source: cats-eo internal; fold framing from Penner — Optics By Example ch. 7, https://leanpub.com/optics-by-example/.
Compute aggregations
A Lens sees one value; a Traversal visits a container's elements.
Indexed addresses a fixed index space, while MultiFocus[F]
supports container traversal and aggregation. The two recipes below
show that distinction; see the MultiFocus reference.
Recipe A — Adjust a fixed-index configuration
Why: a Boolean-indexed configuration has a setting for each coordinate. Read one, rewrite each with its index visible, or fill the whole space without enumerating or comparing indexes.
import cats.Representable
import cats.instances.function.given
import dev.constructive.eo.optics.Indexed
import dev.constructive.eo.data.MultiFocus
import dev.constructive.eo.data.MultiFocus.given
import dev.constructive.eo.data.MultiFocus.{collectList, collectMap}
val readerRepr = summon[Representable.Aux[[a] =>> Boolean => a, Boolean]]
val configGlass = Indexed.representable[[a] =>> Boolean => a, Double, Double](readerRepr)
val config: Boolean => Double = b => if b then 0.5 else 0.2
configGlass.at(true)(config)
// res6: Double = 0.5
val adjustedConfig = configGlass.modify((i, c) => if i then c * 1.4 else c)(config)
// adjustedConfig: Function1[Boolean, Double] = dev.constructive.eo.optics.Indexed$$Lambda/0x000000009c9e8000@4feed5b3
(adjustedConfig(false), adjustedConfig(true))
// res7: Tuple2[Double, Double] = (0.2, 0.7)
val zeroConfig = configGlass.replace(0.0)(config)
// zeroConfig: Function1[Boolean, Double] = dev.constructive.eo.optics.Indexed$$Lambda/0x000000009c9e87b0@5e8e3bc9
(zeroConfig(false), zeroConfig(true))
// res8: Tuple2[Double, Double] = (0.0, 0.0)
This representable constructor returns Indexed.Grate, the
X = Unit specialization. Other glasses retain residual context.
Nested andThen uses product indexes to address the full grid;
Unit-context normalization and generic classical-family seams remain
open. For effectful per-element rewrites, use container traversal.
For nested environment/channel settings, type-changing writes, and labelled sensor grids that retain per-row metadata, continue with Indexed: Grates and Glasses.
Source: Penner — Grate: yet another optic, https://chrispenner.ca/posts/grate.
Recipe B — Summarise a batch, then broadcast or collapse
Why: a column of readings from a batch needs to become a
report: each row pairing the original value with its distance
from the batch average, plus the batch-level statistics — mean,
variance, standard deviation — computed once. The same
MultiFocus[F] optic covers every shape the report needs:
broadcast a summary into every position, collapse the batch to a
single footer value, rewrite every row relative to the batch in
one expression, or fold out the statistical moments in one pass —
you pick the flavour at the call site:
case class ReportRow(value: Double, distanceFromMean: Double)
val readings = List(12.0, 14.0, 11.0, 15.0)
val readingsMF = MultiFocus.apply[List, Double]
// (1) Baseline column — .collectMap broadcasts the batch mean back
// into every position, keeping the shape.
readingsMF.collectMap[Double](xs => xs.sum / xs.size)(readings)
// res9: List[Double] = List(13.0, 13.0, 13.0, 13.0)
// (2) Report footer — .collectList collapses the batch to a single
// summary value, whatever the input length.
readingsMF.collectList(xs => xs.sum / xs.size)(readings)
// res10: List[Double] = List(13.0)
// (3) The report rows, in one expression — .collectWith is the
// algebraic-lens universal: the aggregate sees the whole batch
// ONCE, computes the baseline, and returns the per-row rewrite.
// The pApply factory makes the walk type-changing, so each
// Double becomes a ReportRow pairing the original with its
// distance from the average.
MultiFocus.pApply[List, Double, ReportRow].collectWith { xs =>
val mean = xs.sum / xs.size
v => ReportRow(v, v - mean)
}(readings)
// res11: List[ReportRow] = List(
// ReportRow(value = 12.0, distanceFromMean = -1.0),
// ReportRow(value = 14.0, distanceFromMean = 1.0),
// ReportRow(value = 11.0, distanceFromMean = -2.0),
// ReportRow(value = 15.0, distanceFromMean = 2.0)
// )
// (4) The batch-level statistics for the report footer: count, sum
// and sum of squares in ONE foldMap pass via the tuple monoid,
// then the moments fall out.
val (n, sum, sumSq) = readingsMF.foldMap(v => (1, v, v * v))(readings)
// n: Int = 4
// sum: Double = 52.0
// sumSq: Double = 686.0
val mean = sum / n
// mean: Double = 13.0
val variance = sumSq / n - mean * mean // population variance
// variance: Double = 2.5
val stdDev = math.sqrt(variance)
// stdDev: Double = 1.5811388300841898
Which flavour?
.collectMap[B](agg: F[A] => B)keeps the shape — the summary lands in every position. Reach for it when a downstream step needs the aggregate alongside the originals (share-of-total, a normalisation baseline)..collectList(agg: List[A] => B)supplies a singleton focus vector to reconstruction with the observed context. This identity-backedreadingsMFreturns a singleton List, but other optics can preserve surrounding structure or reject a shape/count mismatch. It is not a general promise of a single-element source..collectWith(agg: F[A] => A => B)is the general map-shaped collect — the aggregate sees the batch once and the returned function runs per position, so batch-relative rewrites (share-of-total, distance-from-mean) are one expression.collectMapis its constant-function special case, and the type-changingMultiFocus.pApply[F, A, B]factory un-pins the element type so the rewrite can emit a different row type (step 3). OnlycollectListstays separate — its singleton collapse changes the focus count, which no map-shaped combinator can express.
Why distinct flavours, rather than one derived automatically? The MultiFocus reference has the answer.
Sources: Penner — Kaleidoscopes: lenses that never die, https://chrispenner.ca/posts/kaleidoscopes; Penner — Algebraic lenses, https://chrispenner.ca/posts/algebraic.
Edit JSON without decoding
Why: you need to change one field deep inside a JSON payload
and pass the rest through untouched — no full decode into a
case-class tree, no re-encode, no decoder for the siblings you
never read. Navigation doesn't stop at the case-class boundary:
codecPrism[S] walks circe's Json directly, and only the focused
leaf is materialised as A:
import dev.constructive.eo.circe.codecPrism
import io.circe.syntax.*
// The payload's case classes — hosted at package level in
// dev.constructive.eo.docs with kindlings-derived codecs (heavy
// derivation macros run in compiled sources, not mdoc fences):
//
// case class UserAddress(street: String, zip: Int)
// object UserAddress:
// given Codec.AsObject[UserAddress] = KindlingsCodecAsObject.derived
//
// case class SiteUser(name: String, address: UserAddress) // ditto
import dev.constructive.eo.docs.{SiteUser, UserAddress}
val userStreet = codecPrism[SiteUser].address.street
val userJson = SiteUser("Alice", UserAddress("Main St", 12345)).asJson
// userJson: Json = JObject(
// object[name -> "Alice",address -> {
// "street" : "Main St",
// "zip" : 12345
// }]
// )
userStreet.modifyUnsafe(_.toUpperCase)(userJson).noSpacesSortKeys
// res12: String = "{\"address\":{\"street\":\"MAIN ST\",\"zip\":12345},\"name\":\"Alice\"}"
The
OrderCirceBench
suite shows this edit is flat in document size — ~1.3 µs whether the record
is small or large — while the decode / .copy / re-encode path scales with the
whole payload, so the gap grows from ~3× on a tiny record to ~160× on a large
one. circe-optics' analogous root.user.address.street surface forces a full
decode per level; cats-eo's cursor walk does not.
The *Unsafe suffix opts out of the failure channel; the default
modify returns Ior[Chain[JsonFailure], Json], so a silent
no-op edit tells you which step refused — the
circe integration's failure flow
covers routing it.
Source: cats-eo internal (JsonPrism); related to circe-optics'
root.* idiom https://circe.github.io/circe/optics.html.
…and every element of a JSON array
Walk an array without materialising it as a Scala collection;
only the focused leaf of each element is decoded. The .each
step splits the Prism into a JsonTraversal:
// Same hosting pattern — kindlings-derived Codec.AsObject on both:
//
// case class Item(name: String, price: Double)
// case class Basket(owner: String, items: Vector[Item])
import dev.constructive.eo.docs.{Basket, Item}
val basket = Basket("Alice", Vector(Item("apple", 1.0), Item("pear", 2.0)))
// basket: Basket = Basket(
// owner = "Alice",
// items = Vector(
// Item(name = "apple", price = 1.0),
// Item(name = "pear", price = 2.0)
// )
// )
val everyItemName = codecPrism[Basket].items.each.name
// everyItemName: JsonTraversal[String] = dev.constructive.eo.circe.JsonTraversal@281100bf
everyItemName.modifyUnsafe(_.toUpperCase)(basket.asJson).noSpacesSortKeys
// res13: String = "{\"items\":[{\"name\":\"APPLE\",\"price\":1.0},{\"name\":\"PEAR\",\"price\":2.0}],\"owner\":\"Alice\"}"
Per-element failures to decode accumulate into
Ior.Both(chain, partialJson) on the default surface — the
circe integration's failure flow
shows how to route the chain.
Source: cats-eo internal (JsonTraversal).
Decoupling and modularity
Because an optic is a concrete, portable value relating two types,
you can replace a direct dependence on a data type with a
dependence on the relationship: a function that needs "every
Instant in whatever you give me" asks for the optic, not the
type. You can also say precisely how much you need — the guarantees
of an Iso, or just the freedom of a fold — and modules that share
no types at all can still interoperate at the call site, or even at
the wire-format level.
Require the optic, not the type
Why: the article's adjustSheetTimes shouldn't have to know
about BalanceSheet. Any function whose real job is "apply f at
every Instant" can take the optic as its parameter and let the
caller decide what structure it runs against — the dependence on
the concrete type evaporates:
import java.time.Instant
import dev.constructive.eo.data.{MultiFocus, PSVec}
def adjustTimes[S](t: Optic[S, S, Instant, Instant, MultiFocus[PSVec]])(
f: Instant => Instant
): S => S = t.modify(f)
Any structure that can point at its Instants gets the behaviour —
all it takes is that structure's own optic. A record whose
timestamps sit in two fields, and a bare collection, are served by
the same function:
case class Meeting(subject: String, start: Instant, end: Instant)
val meetingTimes =
Lens[Meeting, (Instant, Instant)](
m => (m.start, m.end),
(m, se) => m.copy(start = se._1, end = se._2),
).andThen(Traversal.two[(Instant, Instant), (Instant, Instant), Instant, Instant](
_._1,
_._2,
(_, _),
))
val bumpMeeting = adjustTimes(meetingTimes)(_.plusSeconds(3600))
val bumpAll = adjustTimes(Traversal.each[List, Instant])(_.plusSeconds(3600))
bumpMeeting(Meeting("standup", Instant.EPOCH, Instant.EPOCH.plusSeconds(900)))
// res14: Meeting = Meeting(
// subject = "standup",
// start = 1970-01-01T01:00:00Z,
// end = 1970-01-01T01:15:00Z
// )
bumpAll(List(Instant.EPOCH, Instant.EPOCH.plusSeconds(60)))
// res15: List[Instant] = List(1970-01-01T01:00:00Z, 1970-01-01T01:01:00Z)
Meeting and List[Instant] share no interface, no parent trait,
no typeclass — the optic is the interface. This is the article's
central move: interoperability between otherwise wholly independent
modules, done at the call site, with the structure-owning module
publishing one optic value instead of exposing its shape.
Source: Hansen — We Need More Optics, The Startup, https://medium.com/swlh/we-need-more-optics-8ddf1d2d9468 ("Modular Application Design").
Re-use an optic across representations
Why: two subsystems rarely agree on representation — one thinks
in UTC Instants, another in wall-clock LocalDateTimes; one holds
the domain record, another the wire tuple it serialises to. An optic
is a value relating two types, and Optic carries two lawful
cats.arrow.Profunctor instances that let you re-aim a published
optic instead of re-deriving it: Optic.innerProfunctor maps the
focus pair (what the optic yields, and what it accepts back), and
Optic.outerProfunctor maps the source pair (what it reads from,
and what it rebuilds).
Re-aim the focus — meetingTimes above yields Instants, but the
calendar subsystem edits wall-clock times:
import java.time.{LocalDateTime, ZoneOffset}
val meetingWallClock =
Optic
.innerProfunctor[Meeting, Meeting, MultiFocus[PSVec]]
.dimap(meetingTimes)((wall: LocalDateTime) => wall.toInstant(ZoneOffset.UTC))(
LocalDateTime.ofInstant(_, ZoneOffset.UTC)
)
meetingWallClock.modify(_.plusHours(2))(
Meeting("standup", Instant.EPOCH, Instant.EPOCH.plusSeconds(900))
)
// res16: Meeting = Meeting(
// subject = "standup",
// start = 1970-01-01T02:00:00Z,
// end = 1970-01-01T02:15:00Z
// )
Re-aim the source — the scheduling gateway never sees Meeting, only
the (startEpochSecond, endEpochSecond) pair it puts on the wire.
The same optic serves it too:
val wireTimes =
Optic
.outerProfunctor[Instant, Instant, MultiFocus[PSVec]]
.dimap(meetingTimes)((p: (Long, Long)) =>
Meeting("wire", Instant.ofEpochSecond(p._1), Instant.ofEpochSecond(p._2))
)(m => (m.start.getEpochSecond, m.end.getEpochSecond))
wireTimes.modify(_.plusSeconds(3600))((0L, 900L))
// res17: Tuple2[Long, Long] = (3600L, 4500L)
The same moves cover any representation split: UTF-8 vs UTF-16 text
(dimap through the re-encoding pair), big- vs little-endian words
(dimap through java.lang.Long.reverseBytes), or a legacy envelope
around the record you actually care about.
One caveat: dimap returns an anonymous Optic, so the generic
capability surface (.modify, .foldMap, …) applies but
concrete-class fused overloads are erased. When the concrete optic
ships its own input-side mapping — Prism.tearFrom / mendFrom —
prefer that form on hot paths.
Depend only on what's needed
Why: the recipe above still pins the carrier — it accepts
traversals only, and the optic travels as an explicit argument.
Most functions need even less: "something I can fold Doubles out
of", "something I can rewrite Doubles through". The
capability traits — CanGet, CanGetOption,
CanModify, CanFold, … — are exactly those contracts as proper
types, so the optic arrives as using evidence and the subject
type stays fully generic. This is the article's
implicit T: Traversal[T, DateTime] move, with the carrier erased:
import dev.constructive.eo.{CanFold, CanModify}
// Read-side contract: anything that can fold Doubles out of S —
// Lens, Prism, Optional, Traversal, Fold, Iso all qualify.
def total[S](s: S)(using o: CanFold[S, Double]): Double =
o.foldMap(identity)(s)
// Write-side contract: anything that can rewrite Doubles in place.
def scale[S](k: Double)(using cm: CanModify[S, Double]): S => S =
cm.modify(_ * k)
case class Line(desc: String, amount: Double)
case class Invoice(fee: Double, lines: List[Line])
val feeL = lens[Invoice](_.fee)
val lineAmounts =
lens[Invoice](_.lines)
.andThen(Traversal.each[List, Line])
.andThen(lens[Line](_.amount))
Concrete optic classes implement the capabilities, so a lens can
be handed over as the evidence itself; an optic known only at the
generic Optic[…, F] type — like the composed traversal — is bound
as a given and the capability is derived on the spot:
val inv = Invoice(5.0, List(Line("widgets", 10.0), Line("gadgets", 20.0)))
// inv: Invoice = Invoice(
// fee = 5.0,
// lines = List(
// Line(desc = "widgets", amount = 10.0),
// Line(desc = "gadgets", amount = 20.0)
// )
// )
// A Lens IS a CanFold — pass it as the evidence directly.
total(inv)(using feeL)
// res18: Double = 5.0
// The composed traversal is an anonymous Optic; bind it as a given
// and both contracts derive from it.
given Optic[Invoice, Invoice, Double, Double, MultiFocus[PSVec]] = lineAmounts
total(inv)
// res19: Double = 30.0
scale(1.1)(inv)
// res20: Invoice = Invoice(
// fee = 5.0,
// lines = List(
// Line(desc = "widgets", amount = 11.0),
// Line(desc = "gadgets", amount = 22.0)
// )
// )
The trait ladder, weakest first: CanFold (foldMap /
headOption / exists / length / foci), CanGetOption,
CanModify (modify / replace), CanGet, CanReverseGet
(build). A signature that demands only CanFold accepts
everything; one that demands both CanGet and CanReverseGet
insists on an Iso. Under the hood each trait is gated by the
matching typeclass on the carrier (ForgetfulFold[F],
PartialAccessor[F], ForgetfulFunctor[F], Accessor[F],
ReverseAccessor[F]) — a signature can still take Optic[…, F]
plus the gate directly when it genuinely needs the carrier; see
Capabilities for the full matrix, the coherence
rules, and the using-clause ordering footgun that page documents
for hand-written gate signatures.
Source: Hansen — We Need More Optics ("We can eliminate the dependence on the BalanceSheet type by requiring a Traversal instead"; "you are able to specify if you need the guarantees of an Iso, or the freedom of a Getter").
Serdes free pipes
Why: a Kafka consumer gets an Array[Byte] payload, needs to
change one field, and re-emits binary on the producer side. The
conventional answer couples the consumer to the whole schema — a
decode into the full case-class tree per message, on a hot path
where that is pure overhead. AvroPrism decouples it: the module
depends on one field's optic, not on OrderEvent. It IS a
byte-carried optic (Optic[Array[Byte], Array[Byte], A, A,
Affine]): it locates the focused field's byte span under a cached
reader schema, decodes only that slice, and writes by splicing the
re-encoded focus back into the payload — no IndexedRecord
materialised on either side:
import dev.constructive.eo.avro as eoavro
import dev.constructive.eo.avro.AvroCodec
import java.io.ByteArrayOutputStream
import org.apache.avro.generic.{GenericDatumWriter, GenericRecord}
import org.apache.avro.io.EncoderFactory
// Hosted in dev.constructive.eo.docs with kindlings-derived
// AvroEncoder / AvroDecoder / AvroSchemaFor givens:
//
// case class OrderEvent(orderId: String, customer: String, total: Double)
import dev.constructive.eo.docs.OrderEvent
// Stand-in for an inbound Kafka record: serialise an OrderEvent to
// binary under the same schema the prism caches.
val outSchema = summon[AvroCodec[OrderEvent]].schema
val sample = OrderEvent("ord-42", "alice", 99.99)
val sampleBytes: Array[Byte] =
val rec = summon[AvroCodec[OrderEvent]].encode(sample).asInstanceOf[GenericRecord]
val out = new ByteArrayOutputStream()
val encoder = EncoderFactory.get().binaryEncoder(out, null)
val writer = new GenericDatumWriter[GenericRecord](outSchema)
writer.write(rec, encoder)
encoder.flush()
out.toByteArray
// The optic: walk into `customer` once at construction time,
// reuse on every inbound record. Disambiguate the Avro
// `codecPrism` from the circe one imported earlier in this page
// by qualifying through the `eoavro` alias.
val upperCustomer = eoavro.codecPrism[OrderEvent].customer
// Kafka hot path: bytes in → splice in place → bytes out. No
// re-serialisation step — the prism's write side IS the emit path.
val outBytes: Array[Byte] =
upperCustomer.modify(_.toUpperCase)(sampleBytes)
// outBytes: Array[Byte] = Array(
// 12,
// 111,
// 114,
// 100,
// 45,
// 52,
// 50,
// 10,
// 65,
// 76,
// 73,
// 67,
// 69,
// -113,
// -62,
// -11,
// 40,
// 92,
// -1,
// 88,
// 64
// )
// Round-trip witness — decode the output to confirm the customer
// field changed and the rest is preserved.
(upperCustomer.getOption(outBytes),
eoavro.codecPrism[OrderEvent].getOption(outBytes))
// res21: Tuple2[Option[String], Option[OrderEvent]] = (
// Some("ALICE"),
// Some(OrderEvent(orderId = "ord-42", customer = "ALICE", total = 99.99))
// )
.modify on the byte optic is silent-pass-through — bad bytes,
missing fields, or decode mismatches leave the input payload
untouched (an Affine Miss) rather than allocating an Ior
chain. That matches the Kafka consumer budget: at-least-once
delivery already implies the consumer must be tolerant of
malformed payloads at the offset commit boundary, and the
per-record allocation cost of Ior is a tax on the happy path.
When you DO want the diagnostic — for a dead-letter queue, say —
flip to the record-carried face:
upperCustomer.record.modify(...) accepts the same bytes and
returns Ior[Chain[AvroFailure], IndexedRecord]; route on the
Ior.Both / Ior.Left shape from there.
The cached reader schema is the load-bearing piece: a single
codecPrism[OrderEvent] value pins the schema once, and the
parser reuses it across millions of inbound records. For the
schema-registry case where the reader schema arrives at
runtime, use the explicit-schema overload —
AvroPrism.codecPrism[OrderEvent](runtimeSchema) — to bypass
the kindlings-derived schema entirely.
Source: cats-eo internal (AvroPrism's byte-carried
default surface). Background framing on the streaming /
Kafka use case lives in the
Avro integration intro.
Inspectable cross-format bridges
Why: the consumer reads Avro off a topic and must emit JSON
to a partner API — or receives JSON and must re-emit Avro. Two
modules, two wire formats, and the only thing they genuinely share
is one field. The conventional bridge couples them anyway: decode
the whole record into a case class, build a whole output value,
encode it all. With a byte optic on each side of the seam, only the
moved branch is ever decoded and re-encoded: the full object is
never constructed on either format. (AvroJsonBridgeSpec
proves this with a counting root codec — both bridge directions run
with zero root decodes and zero root encodes; AvroJsonBridgeBench
puts B/op numbers on it.)
import com.github.plokhotnyuk.jsoniter_scala.core.JsonValueCodec
import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker
import dev.constructive.eo.jsoniter.JsoniterPrism
given JsonValueCodec[String] = JsonCodecMaker.make
// One byte optic per format, same focus type. Drilled once,
// reused for every message. (`JsoniterPrism.fromPath` returns an
// `Either` — a path is data; the literal below is unwrapped here.)
val customerAvro = eoavro.codecPrism[OrderEvent].customer
val customerJson =
JsoniterPrism
.fromPath[String]("$.customer")
.fold(msg => throw new IllegalArgumentException(msg), identity)
// The JSON side's output skeleton. Placeholders must be VALID
// encodings of the branch type — the splice write decodes the
// current focus before replacing it.
val receiptTemplate: Array[Byte] =
"""{"kind":"receipt","customer":"","status":"ok"}""".getBytes("UTF-8")
// Avro → JSON: slice the branch off the Avro wire form, splice
// it into the JSON template. No OrderEvent, no receipt object.
val receipt = customerAvro.getOption(sampleBytes) match
case Some(c) => customerJson.replace(c)(receiptTemplate)
case None => receiptTemplate
// receipt: Array[Byte] = Array(
// 123,
// 34,
// 107,
// 105,
// 110,
// 100,
// 34,
// 58,
// 34,
// 114,
// 101,
// 99,
// 101,
// 105,
// 112,
// 116,
// 34,
// 44,
// 34,
// 99,
// 117,
// 115,
// 116,
// 111,
// 109,
// 101,
// 114,
// 34,
// 58,
// 34,
// 97,
// 108,
// 105,
// 99,
// 101,
// 34,
// 44,
// 34,
// 115,
// 116,
// 97,
// 116,
// 117,
// 115,
// 34,
// 58,
// 34,
// 111,
// ...
new String(receipt, "UTF-8")
// res22: String = "{\"kind\":\"receipt\",\"customer\":\"alice\",\"status\":\"ok\"}"
// JSON → Avro: read the branch from JSON bytes, splice it into
// existing Avro wire bytes. Same guarantee, reversed.
val inbound: Array[Byte] =
"""{"kind":"receipt","customer":"carol","status":"ok"}""".getBytes("UTF-8")
// inbound: Array[Byte] = Array(
// 123,
// 34,
// 107,
// 105,
// 110,
// 100,
// 34,
// 58,
// 34,
// 114,
// 101,
// 99,
// 101,
// 105,
// 112,
// 116,
// 34,
// 44,
// 34,
// 99,
// 117,
// 115,
// 116,
// 111,
// 109,
// 101,
// 114,
// 34,
// 58,
// 34,
// 99,
// 97,
// 114,
// 111,
// 108,
// 34,
// 44,
// 34,
// 115,
// 116,
// 97,
// 116,
// 117,
// 115,
// 34,
// 58,
// 34,
// 111,
// ...
val updatedAvro = customerJson.getOption(inbound) match
case Some(c) => customerAvro.replace(c)(sampleBytes)
case None => sampleBytes
// updatedAvro: Array[Byte] = Array(
// 12,
// 111,
// 114,
// 100,
// 45,
// 52,
// 50,
// 10,
// 99,
// 97,
// 114,
// 111,
// 108,
// -113,
// -62,
// -11,
// 40,
// 92,
// -1,
// 88,
// 64
// )
// Witness (this read DOES construct the object — the bridge
// above never did): the branch moved, the siblings survived.
eoavro.codecPrism[OrderEvent].getOption(updatedAvro)
// res23: Option[OrderEvent] = Some(
// OrderEvent(orderId = "ord-42", customer = "carol", total = 99.99)
// )
Cost model worth knowing: each .replace locates the span,
decodes the current focus (the Affine write carries it),
re-encodes the new one, and allocates one output buffer. Moving
one branch this way is far cheaper than materialising a wide
object — but for many branches per message, or fragment moves
between two Avro payloads, prefer sliceBytes / graftBytes
(no decode at all).
Source: AvroJsonBridgeSpec (avro module, eo.avro.jsoniter) and
AvroJsonBridgeBench (benchmarks) in cats-eo.
Re-usable laws for testing
A decoupled boundary needs a tested contract. Composing optics
means the combinators are already tested — what's left to check is
your instances, the optic values your module publishes. The
discipline suites that gate cats-eo's own carriers ship in
cats-eo-laws, so a composed optic like meetingTimes gets the
full Traversal rule-set in one checkAll instead of hand-written
round-trip tests:
libraryDependencies += "dev.constructive" %% "cats-eo-laws" % "0.18" % Test
import cats.Functor
import dev.constructive.eo.laws.TraversalLaws
import dev.constructive.eo.laws.discipline.TraversalTests
// meetingTimes walks a plain Meeting, so T is the constant type
// lambda — its Functor is one line. Arbitrary[Meeting] and
// Cogen[Instant] come from your ScalaCheck generators.
given Functor[[X] =>> Meeting] with
def map[A, B](fa: Meeting)(f: A => B): Meeting = fa
checkAll(
"meetingTimes",
new TraversalTests[[X] =>> Meeting, Instant]:
val laws = new TraversalLaws[[X] =>> Meeting, Instant]:
val traversal = meetingTimes
.traversal,
)
Every optic family has a matching FooLaws / FooTests pair — the
migration guide has
the import-swap table.
Source: Hansen — We Need More Optics, The Startup, https://medium.com/swlh/we-need-more-optics-8ddf1d2d9468 ("Re-usable tests").
Effect threading
Optics are great at turning a host language's control flow into
just type definitions. The optic fixes where — which foci, at
what depth; the effect type G you thread through .modifyF /
.modifyA decides what the control flow is: abort on first
failure, accumulate every failure, log alongside the rewrite, defer
into IO. Swapping behaviours means swapping G, never touching
the optic.
Validate-in-place with modifyF
Why: the update can fail. Bump age by 1, but reject a
negative input with None — and keep the validation and the write
as one expression instead of a get / check / set dance. .modifyF[G]
lifts an A => G[B] through any carrier that admits Functor[G]:
case class Visitor(name: String, age: Int)
val visitorAgeL = lens[Visitor](_.age)
import cats.syntax.functor.*
import cats.instances.option.*
visitorAgeL.modifyF[Option](age =>
if age >= 0 then Some(age + 1) else None
)(Visitor("Alice", 30))
// res24: Option[Visitor] = Some(Visitor(name = "Alice", age = 31))
visitorAgeL.modifyF[Option](age =>
if age >= 0 then Some(age + 1) else None
)(Visitor("Alice", -1))
// res25: Option[Visitor] = None
.modifyA[G] is the Applicative[G] variant — reach for it
when the chain has branching effects to combine (traversal +
validation, for instance); the next recipe is exactly that.
Witherable-style filter-and-drop traversal is cross-referenced from
Penner — Composable filters using Witherable optics;
the carrier is deferred to a follow-up release.
Source: Monocle / Haskell lens classic traverseOf,
generalised.
Structure orthogonal to effects
Why: the article's systemVerifier insight — "what to do is
decided by the effect type" — in its smallest honest form. Take the
lineAmounts traversal from the "Depend only on what's needed"
recipe and thread three different Gs through the same
.modifyA. The
optic never changes; the control flow does:
import cats.data.{ValidatedNel, Writer}
import cats.syntax.validated.*
type Checked[A] = ValidatedNel[String, A]
type Audit[A] = Writer[List[String], A]
val badInv = Invoice(5.0, List(Line("widgets", -10.0), Line("gadgets", -20.0)))
// G = Option: abort on the first bad focus — short-circuit control flow.
lineAmounts.modifyA[Option](a =>
if a > 0 then Some(a * 1.1) else None
)(badInv)
// res26: Option[Invoice] = None
// G = ValidatedNel: visit EVERY focus, accumulate every failure —
// same optic, same lambda shape, completely different control flow.
lineAmounts.modifyA[Checked](a =>
if a > 0 then (a * 1.1).validNel else s"non-positive amount: $a".invalidNel
)(badInv)
// res27: Validated[NonEmptyList[String], Invoice] = Invalid(
// NonEmptyList(
// head = "non-positive amount: -10.0",
// tail = List("non-positive amount: -20.0")
// )
// )
// G = Writer: rewrite AND emit an audit line per focus — the effect
// threads a log through the walk without touching the domain types.
lineAmounts.modifyA[Audit](a =>
Writer(List(s"scaled $a"), a * 1.1)
)(inv).run
// res28: Tuple2[List[String], Invoice] = (
// List("scaled 10.0", "scaled 20.0"),
// Invoice(
// fee = 5.0,
// lines = List(
// Line(desc = "widgets", amount = 11.0),
// Line(desc = "gadgets", amount = 22.0)
// )
// )
// )
The same slot takes IO (each focus becomes an effect to
sequence), Eval (defer the whole rewrite), or an fs2 Stream —
the article's original systemVerifier pipes every focus of a
traversal through a verification stream with .modifyF, and its
shape is exactly the three calls above with a fancier G. The
domain types never learn any of this is happening.
Source: Hansen — We Need More Optics, The Startup, https://medium.com/swlh/we-need-more-optics-8ddf1d2d9468 ("What to do is decided by the effect type").
Batch-load nested IDs
Why: every node in a structure carries an ID you need to
resolve against a database, and the naive .modifyA fires one
query per node — the classic N+1. Use the same traversal twice:
foldMap collects every ID in one pass, you issue a single batched
query, then .modify distributes the results back. 100×–300×
fewer queries with no change to the domain types:
case class Node(id: Int, label: String)
case class Payload(id: Int, body: String)
// Stand-in for a DB-backed batch fetch.
def fetchAll(ids: List[Int]): Map[Int, Payload] =
ids.map(i => i -> Payload(i, s"body-$i")).toMap
val eachLeaf = Traversal.each[List, Node]
val nodes = List(Node(1, "a"), Node(2, "b"), Node(3, "c"))
// nodes: List[Node] = List(
// Node(id = 1, label = "a"),
// Node(id = 2, label = "b"),
// Node(id = 3, label = "c")
// )
// Pass 1: collect every ID into a single list via foldMap.
val allIds = eachLeaf.foldMap((n: Node) => List(n.id))(nodes)
// allIds: List[Int] = List(1, 2, 3)
// Pass 2: issue ONE query for the whole set.
val byId = fetchAll(allIds)
// byId: Map[Int, Payload] = Map(
// 1 -> Payload(id = 1, body = "body-1"),
// 2 -> Payload(id = 2, body = "body-2"),
// 3 -> Payload(id = 3, body = "body-3")
// )
// Pass 3: broadcast the resolved payloads back through the same
// traversal. The structure is preserved; only the labels shift.
eachLeaf.modify(n => n.copy(label = byId(n.id).body))(nodes)
// res29: List[Node] = List(
// Node(id = 1, label = "body-1"),
// Node(id = 2, label = "body-2"),
// Node(id = 3, label = "body-3")
// )
The pattern generalises to trees, graphs, nested containers,
anything with a Traverse. The two-pass idiom is what cats-eo's
foldMap + modify pair already enables out of the box — no
cats-eo-specific API to learn.
Source: Penner — Using traversals to batch database queries, https://chrispenner.ca/posts/traversals-for-batching.
Persist-and-stamp decode free
Why: a PUT (or POST) handler receives a draft entity on the
wire, persists it, and must return the same entity enriched with
the database-assigned id. The id only exists after the effectful
store runs, so this is the classic "set one field to the result of
an effect" shape — and a derived id-lens makes the stamp a one-liner.
With circe carrying the value across the wire on both ends, the
whole handler is a short pipeline: decode → store → stamp →
encode.
import io.circe.Json
import io.circe.syntax.*
import io.circe.parser.decode
import dev.constructive.eo.generics.lens
// cats.Eval stands in for your effect type (cats-effect IO, ZIO,
// Future…) — deferred, has map/flatMap, already on the classpath.
import cats.Eval
// Hosted in dev.constructive.eo.docs with a kindlings-derived
// Codec.AsObject given:
//
// final case class BalanceSheet(id: Long, owner: String, total: Double)
import dev.constructive.eo.docs.BalanceSheet
// One derived lens onto the id — no hand-written getter/setter, and
// it works even though `id` shares the case class with two other fields.
val sheetId = lens[BalanceSheet](_.id)
// The effectful store: inserts the row, hands back the generated id.
def save(sheet: BalanceSheet): Eval[Long] = Eval.always {
val _ = sheet // pretend: INSERT … RETURNING id
42L
}
// Persist, then stamp the returned id back onto the object with the
// lens — `save(...).map(sheetId.replace(_)(sheet))`.
def store(sheet: BalanceSheet): Eval[BalanceSheet] =
save(sheet).map(sheetId.replace(_)(sheet))
// The whole PUT handler: request bytes in, response bytes out.
def handlePut(body: String): Eval[String] =
decode[BalanceSheet](body) match
case Right(draft) => store(draft).map(_.asJson.noSpaces)
case Left(err) => Eval.now(Json.obj("error" -> err.getMessage.asJson).noSpaces)
// Incoming PUT body — `id` is a placeholder the store overwrites.
val request = """{"id":0,"owner":"Acme Corp","total":1234.5}"""
// request: String = "{\"id\":0,\"owner\":\"Acme Corp\",\"total\":1234.5}"
handlePut(request).value
// res30: String = "{\"id\":42,\"owner\":\"Acme Corp\",\"total\":1234.5}"
Every step earns its place: decode is the wire → domain hop
(circe), store is your effect, the stamp is the derived lens
standing in for a hand-written sheet.copy(id = newId), and
asJson is the domain → wire hop back out. The lens is the only
optic, and it stays a reusable value — compose it further
(orderLens.andThen(sheetId)) the moment the entity nests inside a
larger request. In production you'd reach for an opaque DbId
rather than a bare Long, and likely a separate Draft type
without the id field; the shape of the pipeline doesn't change.
Source: the lens-stamps-the-effect-result pattern, https://gist.github.com/kryptt/af0a626849f0e3f5b16fcd161a5545e4; see also Generics → Composing into pipelines.
Further reading
- Concepts — the theory behind the unified Optic
trait and carriers; the
composition lattice
diagram maps out every
Composer[F, G]bridge. - Optics reference — the full per-family tour, introduced by the family taxonomy diagram.
- Generics — macro-derived
lens[S](...)andprism[S, A]. - Circe integration — cursor-backed JSON optics; failure flow for the Ior decision tree.
- Avro integration — cursor-backed Avro optics; failure flow for the schema-driven Ior decision tree.
- Migrating from Monocle — side-by-side translation guide.
- Hansen — We Need More Optics, The Startup, https://medium.com/swlh/we-need-more-optics-8ddf1d2d9468 — the article this page's three-part structure follows.