It’s interesting, SICP and other many other “classic” texts talk about designing programs, but these days I think the much more important skill is designing systems.
I don’t know if distributed systems is consider part of “Computer Science” but it is a much more common problem that I see needs to be solved.
I try to write systems in the simplest way possible and then use observability tools to figure out where the design is deficient and then maybe I will pull out a data structure or some other “computer sciency” thing to solve that problem. It turns out that big O notation and runtime complexity doesn’t matter the majority of the time and you can solve most problems with arrays and fast CPUs. And even when you have runtime problems you should profile the program to find the hot spots.
What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.
The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.
Maybe I am just biased because I have a computer/electrical engineering background.
> but these days I think the much more important skill is designing systems.
It is hard to design systems if you don't have the perspective of implementing them. Yes, you move up the value chain to designing things, no, but no, you don't get to skip gaining experience lower down the value chain.
> What computer science doesn’t teach you is how memory caching works in CPUs.
That was literally my first quarter in my CS undergrad 30 years ago, the old Hennessy and Patterson book, which I believe is still used today. Are things so different now?
> The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.
All of that was covered in my CS undergrad, I wasn't even in a fancy computer engineering/EE background.
I think CS 30 years ago was closer to computer engineering today.
At my uni 10 years ago the CS program didn’t touch anything related to hardware, hell the CS program didn’t even need to take multivariable calculus. In my computer engineering program we covered solid state physics, electromagnetism, digital electronics design, digital signals processing, CPU architecture, compiler design, OS design, algorithms, software engineering, distributed systems design.
The computer engineering program took you from solid state physics and transistor design to PAXOS.
The CS program was much more focused on logic proofs and more formalism and they never touched anything hardware adjacent.
I realize this is different between programs, but from what I read and hear many CS programs these days start at Java and never go down abstraction levels.
I do agree with you that learning the fundamentals is important, but I would argue that a SICP type course is not fundamental — physics is fundamental. And once you learn how we use physics to build CPUs you learn that fancy algorithms and complex solutions are not necessary most of the time given how fast computers are today. If you can get your CPU pipelined properly with high cache hits, branch prediction hits, prefetch hits, and SIMD you can easily brute force many problems.
And for those 10% of problems which cannot be brute forced, 90% of those problems can be solved with profiling and memoization, and for the 10% of those problems you cannot solve with memoization you can solve 90% of them with b-trees.
A top tier CS program is going to make you learn computer architecture along side automata and proofs. MIT went the extra mile with the SICP, it was honestly a hole I didn’t have access to in my top tier program, but I only realized this because I studied PL in grad school. You should go through it if you haven’t, I think it would have made my ugrad experience better and I definitely benefited from a great well rounded curriculum already (UW CSE is still no slouch, but it isn’t MIT!).
If you are into physics and mechanics, then you have to check the SICM (SICP’s less famous cousin) out as well. Again, MIT went the extra mile with that as well.
Computers today are slower than they have ever been. And tomorrow's computers are going to be even slower.
In many applications, the amount of data grows at least as quickly as computer performance. If the time complexity of an algorithm is superlinear, today's computer needs more time to run it with today's data than yesterday's computer did with yesterday's data. Algorithms that used to be practical get more and more expensive to run, until they eventually become impractical.
The more data you have, the more you have to think about algorithms. Brute-forcing can be expensive in terms of compute costs and wall-clock time, while low-level optimizations can take a lot of developer time.
if we're talking about relative human-experienced performance, the slowest computers i ever owned were in the early-mid 00s. they sped up as multicore and ssd's entered the picture and plateaued about ten years ago ime.
The most striking performance observation of my experience was that Apple took a system, OpenStep 4.2 which ran okay on a 33 MHz 68040 (and acceptably on my 25MHz Cube) and made it run only a little bit better on a 400MHz G3 as the Mac OS X Public Beta.
The difference of course was anti-aliasing, and much greater bit depth, and running multiple programming environments/toolkits (Carbon and Java).
We (at a public research university in the US) designed a rudimentary CPU, wrote mips assembly, and understood computer architecture for our CS degree. I graduated 6 years ago
Edit: we also did formal methods and proofs as part of core curriculum
Concur. 15 years ago, as sophomores we built a functional virtual cpu from the transistor level up. As I recall that was about 4 weeks, for the rest of the course, we extended the CPU with cache, extended opcode support, bus expansion, implemented a lexer/parser and ended writing our own assembly subset implementation and using that to write programs.
As for the OP's contention that computer science doesn't teach you to look for higher level things like cache thrashing, I wholeheartedly dissent with that supposition.
I recall at least 3 courses where substantial coursework was devoted to hacking, stack smashing, reverse compiling, profiling/introspection, kernel modification, so much beyond 'dynamic typing is an OO antipattern' stuff that gets IMO erroneously conflated with CS degrees.
Maybe these shit schools exist, but in a top 20 program you will definitely learn cache pitfalls.
There are still jobs where people write frameworks, database engines or version control tools. Those jobs require heavy CS and algorithms, data structures day to day. But there are less of those jobs nowadays as no one is implementing db engine for their app they just use Postgres.
Other jobs that is vast majority is dealing with implementing business logic. Using database with understanding how it works in details is of course going to produce better outcomes. Yet one still can produce great amount of working software without knowing how indexes are stored on disk.
Also a lot of CS graduates fell into a trap where they think their job is to write a framework - where in reality they should just use frameworks and implement business logic- while using CS background to fully understand frameworks already existing.
> while using CS background to fully understand frameworks already existing.
Most frameworks today are so complicated that you typically cannot understand them fully, and even understanding them somewhat partially is more than a full-time job.
I wish someone told me this back when I was trying to get a programming job as a self taught programmer. I would do things like try to build a simple React clone thinking it would help me overcome imposter syndrome to fully understand things from the base up, but it was pretty futile because no one really has time to wrap their head around something that big unless they are paid full time to do it.
You don't have to understand them fully to make use of them. One has to understand design patterns and underlying reasoning, understand context of the framework like for example if it is JS framework that it runs in browser to understand which parts are "because we are running in the browser" vs "that is just why framework implemented it" and if it is Typescript then how it blends into that mix.
Then for any details or unexpected behavior knowing where to look in documentation.
> Yet one still can produce great amount of working software without knowing how indexes are stored on disk.
I agree... up to a point. Most software will likely be replaced/obsolete before it even reaches a scale where indexes even matter (at all) given how fast the underlying hardware is at this point.
... but I don't think this is particularly relevant wrt. the "to CS or not CS" question. If a CS grad has been paying any attention they usually have a decent idea of what kinds of problems are intractable vs. problems that are tractable (but maybe expensive to compute) vs. easy. Also just general exposure to different ways to approach solving a problem (logic programming, pure functional, etc.) can be very valuable. There's just much that one couldn't come up with on their own if one weren't exposed to the ideas from the vast expanse of ideas that are known in CS. (And even a master's doesn't come close to scratching the surface of it all.)
>What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.
Traditionally, the field of databases is largely about solving algorithm problems in the scenario where you have much more data that can fit in memory. Data exists on disk as "pages", you have a fixed number of "page slots" in RAM. Moving pages from disk to RAM or RAM to disk is slow, so you want to do as little of that as you can. This makes trivial problems interesting -- e.g. there's no notion of a 'join' in classic computer science because it's too trivial to bother naming.
We're used to thinking of the study of algorithms as a sort of pure essence, but one could argue that algorithmic efficiency is only meaningful in a particular data and hardware context. That's part of what keeps our jobs interesting, I guess -- otherwise algorithm expertise wouldn't be as useful, since you could just apply libraries/cookbook solutions everywhere.
True. However I find that most junior and even experienced programmers struggle with tactical level coding. I'm really suffering with this right now because the small component I'm tasked with making a small change to is annoyingly stateful and deals with 2 abstractions at once. (It processes files and uses the file system and database to store it's state). I'm shocked how badly it has been thought out. I've spent days trying to avoid doing what has gone before, bits bolted on that make it even more difficult to understand. It really seems that pull request culture has just led to any old crap being approved because no one has the band width to think deeply about the actual code. Bring back in person code reviews !
Unfortunately, while I really want to love Software Design for Flexibility, it's clear that Hanson and Sussman haven't really solved (or even come close to solving) the problem they have identified in the book.
The introduction to that book is brilliant at identifying just how much room software has to grow (you can find similar talks from various Strange Loop sessions Sussman has done), and is really quite inspirational for anyone seriously thinking about the future of computing.
But the rest of the book fails to provide a coherent answer to the questions that are brought up in the intro. It shows off some neat functional programming tricks, but repeatedly fails to deliver on solving the (admittedly ambitious) challenges it provides for itself.
I'm still glad I have a copy, and have re-read the first half multiple times now, but sadly it's not the book it wants to be. To be fair though, that is because we haven't come close to understanding computation enough to solve those problems.
It's a very ambitious book that falls short of it's own ambitious.
Written as the textbook for a software engineering course, it developed out of that course being taught multiple times _and_ all the code reviews which that entailed.
Previous discussions/mentions here which had a notable number of comments:
I re-wrote my current project in the course of reading it (I would read a chapter, then read through the code and where appropriate apply the relevant principle) and once I finish the current re-write (from OpenSCAD to Python) will be repeating that process to see if what I was supposed to have learned stuck/survived the re-write.
IMO it's not excellent. It's not like SICP, it's obtuse for no reason, I find it a hard slog. Flexibility is good but it seems to try to make every bit of your program flexibile and pluggable and you just need to do something eventually.
My opinion, I'd welcome others on the book; there was a small splash when it came out but not much discussion since.
I haven't read the book, but my experience is that the way to make things flexible is to make them simple as possible.
When I've used (or built) something that was built in the style like you're talking about, it's almost always wrong, and the extra complexity and stuff now makes it harder to do right. It's not surprising: unknown future requirements are unknown. Over building is trying to predict the future.
It's like someone building a shed and pouring a foundation that can work for a skyscraper. Except it turns out what we needed was a house that has a different footprint. Or maybe the skyscraper is twice the height and has a stop for the newly-built underneath. Now we have to break apart the foundation before we can even begin work on new stuff; it would have been less work if the original just used a foundation for a shed.
I think that is what the book does. Step by step introducing new requirements, by telling you about a situation, where the previous code would not be flexible enough. I am sure it doesn't state, that one should apply all of its ideas all the time, regardless of the project at hand.
I don't know about books, but I think the best approach is functional programming in a dynamic language. That could be because I'm currently an Elixir fanboy, but I think Lisps, especially Scheme or Clojure, or a functional-restricted approach in JavaScript could do it as well. I agree with parent comment that it's better to keep things as simple as possible and make the changes when necessary vs. building in all the flexibility in the beginning.
Nicely said. The way I think about it: if we can’t write legible and adaptable functions, then we have no chance at making viable systems. All the same engineering skills are at play, just on a different scale.
I find books like SICP interesting and not very useful. I love reading them because I like this stuff, but I don’t get to apply their teachings in real world software. It’s a problem because naturally I want to spend my time reading these kind of books, but if I do that I would be jobless. I need to divide my time between reading pearls like SICP and boring Kafka/Postgres/Golang/K8s/AWS documentation.
I don’t find them useful in the sense of directly applying practical techniques in my day job, but I consider them somewhat necessary background reading to get into the right state of mind. You can very quickly tell when someone never acquired any academic knowledge in this area (or never played with functional languages or similar pastimes) - you can’t explain to those people why modifying global variables all over the place in a large program is a bad idea and other things like that. They just nod along skeptically and then somehow keep stumbling into the same kind of mess over and over.
You kind of defeat your own argument. You say it's important to learn "academic knowledge", but then acknowledge the organization will not value your knowledge.
Well in my experience good organizations do recognize that the better design means lower costs in the long run, and people who don’t get that tend to not get promoted. Communicating this effectively up and down the chain is a whole different art in itself though.
One of the problems I've seen is that when new learners and self-taught individuals ask for advice, a lot of software engineers give recommendations based on what they wish their job was or how they would like to imagine themselves.
This is a real problem that people with experience put on learners. If you asked them how they learned it, they'd tell you an entirely different story about starting small, practicing often, trying many ideas, having a strong motivation and some occasional guidance. But they tell others to follow a rigorously defined path which creates the opposite mindset that a learner needs.
The first reason why I really loved SICP is that it is based on Scheme, a language with powerful primitives. I came from a self-taught world of PL/1, Algol, C, then later C++, Java etc. None of them had closures, hygienic macros, anonymous functions, functional programming, call/cc, and of course, "amb", the non-deterministic choice operator. At an even more basic level, SICP taught me that a lot of non-trivial code can be written with just sequences and maps, with good enough efficiency!
Because SICP's starting point was so high, they could describe many concepts easily from the ground up, from object oriented programming, backtracking, constraint programming and non-determinism.
This taught me a number of techniques to apply in real-life, because I could readily identify the missing building blocks in the language or system I was given to work with. For example, I was able to build a lightweight threads system in Java quite readily because I knew that the missing piece was a continuations feature in Java.
> I think the much more important skill is designing systems
Systems engineering is a seperate discipline in engineering fields, same applies to CS I would think, specifically if you look at what most developers do as computer engineering it would only make sense that there is forward progression within the discipline that builds on the foundations tought in most CS courses.
Regarding CS not teaching caching, well any course writer in a CS program that doesn't at least touch on that should feel like they failed their students, it is really something fundamental that should be tought pretty soon.
I feel like profiling should be something that is tought somewhere in a CS program, even a half semester course can dramatically improve peoples understanding of what the machine is actually doing.
On that note, even if you never intend to use the language, watch some C++ conference talks, almost every year at every conference thwre is a talk discussing performance.
> big O notation and runtime complexity doesn’t matter the majority of the time and you can solve most problems with arrays
I have the exact opposite experience.
Software comes out best if you always ensure to use an approach with sensible runtime complexity, and only make trade-offs towards cache-friendly-worse-O implementations where you benchmarked thoroughly.
Most cases where I encounter mega slow programs are because somebody put in something quadratic instead of using a simple, standard O(n logn) solution.
The right book for the right problem. SICP isn't meant to teach you how to tackle fault-tolerance in a complex distributed system. Here is a textbook that talks about distributed systems (van Steen and Tannenbaum):
> What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.
Computer architecture and organization should teach this, no?
> What computer science doesn’t teach you is how memory caching works in CPUs.
Yes it can, and there are tons of papers about data structures to use in various scenarios to handle not just L1, L2, L3, but also NUMA. Sure, this isn’t in SICP, but claiming CS as a field completely ignores how memory works is incorrect.
There are still innumerable people writing standalone programs, single-purpose embedded systems, independent components and libraries, etc
The industry has expanded to include a lot of large-scale distributed cloud projects (often where we might have expected mainframes and cobol before), with many of today's largest employers doing most of their work there, but none of that other stuff really went away. It's still being done every day.
You need a book for what you're doing, and not every book is going to be that. Apparently, SICP is not it. I possess and have read many books, and only some small number of them are applicable to the projects I'm working on at any time.
They don't compete with each other, they complement each other.
If you knew how to design programs you could run it all on a single box and wouldn’t have to design “systems.”
I’m being slightly facetious, but only slightly. If you really think everything is solvable with arrays, you are not going to scale well and of course you’re going to need to throw a lot more hardware at the problem.
My argument is that 90% of problems can be solved with arrays, 5% of problems can be solved with memoization, 3% of problems can be solved with b-trees, and 2% of problems with other data structures.
It is good to know that solutions to the 2% exists, but what we should be focusing on is writing the simplest code possible which solves the problem and then only optimize afterwards using a profiler.
God forbid you have to work on some codebase written by someone who believes they are the second coming of haskell with crazy recursion and backtracing, monads, red black trees, and a DSL on top of the whole thing.
You are right that many problems can be solved with a single box, but my argument is that you do not need fancy algorithms to solve problems on a single box. We should strive to use single boxes whenever possible to reduce complexity.
Computation is designed by humans to serve humans, we should make it as easy as possible for humans to understand. I’m probably going to start a flamewar here, but this is why simple solutions like UNIX and golang have prevailed in the past. Simple code is easy to understand and therefore it is easy to modify and reason about. Some people think simple means that you decompose programs into the smallest possible functional parts, but simple to me is a 500 line main function.
My CS degree covered hardware architecture, disk design, OS design, and also system design, alongside algorithms, data-structures, various programming language and paradigm, etc.
I hadn't seen a blessed PDF version until today. Circa 2001, only the HTML version was freely available, and someone converted it to TeXinfo: https://www.neilvandyke.org/sicp-texi/
For anyone wishing to try: the maintainers of MIT Scheme no longer provide a .dmg but you can download and build the x86_64 version of MIT Scheme. The current release (v12.1) works on a Mac running Sequoia with Intel CPU or on Apple silicon via Rosetta. But the native code compiler (not necessary for SICP AFAIK) is a little broken. (Anecdotally it worked on macOS prior to Monterey, so maybe an Apple-supplied dependency changed. Haven't tracked down the issue.)
All of that is to say: if you do not need MIT Scheme and don't want to fuss with compiling it, then Racket might be a better way to go.
There isn't an active maintainer any more, I'm afraid. And Apple placed restrictions on modifying memory that contains instructions that prevented techniques the compiler relies upon for fast allocation of closures. There are ideas for workarounds, but they would require a lot of work.
I'm a huge fan of MIT Scheme, and have used it since 1984, but I would recommend using another implementation these days, especially on Mac.
Just as a data point, I'd recommend going through it in Racket, which I believe has an explicit SICP mode. I went through it in GNU Guile and it was a pain because there were some minor syntactic differences between Guile and MIT Scheme.
The texinfo version was I believe the source for the really nice HTML5 version if you want to read it in a browser, but with nice formatting that the MIT original version: https://sarabander.github.io/sicp/
one thing to note is that the second chapter's "picture language" is not supported in MIT Scheme in 2024. There used to be a package but it's like 2 decades out of maintenance. In Dr. Racket however, there is a package specifically for working through those problems.
I haven’t seen it in the comments yet, but you can watch Abelson and Sussman teaching the material from this book from recorded lectures in 1986.
I still find their description of how to create and group abstractions in various layers to be useful personally and as a mentor. (In the videos, lesson 3A, 1:07:55)
Just dropping in to say that The Elements of Programming Style is worth reading three times — and I have read it many more times than that, and benefitted from it. Here's my review (from 2010) if you're interested: https://reprog.wordpress.com/2010/03/06/programming-books-pa...
My favourite part of SICP and something that has stuck with me for years is the idea of "wishful programming". That is where you build something top-down by simply wishing you had the lower-level routines. Then, of course, you actually go and build those lower-level routines until you reach the bottom. I find this way of thinking works really well with test-driven development. Write a test against functionality you wish you had, then go and fulfill that wish. Most developers seem to build stuff bottom-up and then end up with something that isn't really what anyone wished for.
They do that because their wish is performance and naturalness.
You may accidentally wish something you don’t yet know the true nature of, and this will create a fragile mess at the bottom. It usually does, cause algorithmic nature of things is rarely intuitive. Starting from the bottom is like starting from quarks that you have rather than from “I want magic to exist”. Well it does not. You reach the bottom and there’s quarks instead of magicules and you’ve lost all context clues on the way which could help to convert between two physics.
Both approaches have their use, because sometimes you have to be bold with your wishes to solve a deep problem. But personally I prefer magic to be packed into the before-topmost layer. I.e. build from the bottom up, and then, just before the business logic, create a convenience magic layer that translates to/from business speak. It becomes adjustable and doesn’t induce a tangled mess all the way down.
So I think one of the things best avoided in life in general is extremity, in all its various guises. No single technique should be followed like scripture, but rather incorporated into ones toolkit and used where appropriate. Building top-down will get you where you want, but risks fragile underpinnings due to a lack of cross-cutting architectural guidance. But, on the other hand, bottom-up might get you the best foundations at each layer but ultimately deliver nothing of value to the users. In practice it's necessary to take a balanced approach and, of course, mistakes will be made and experience will become the guide. Like you I definitely employ the "meet in the middle" approach in practice, but what SICP taught me is how to think about starting at the top, that is, to build upon wishes.
Interestingly, Dr. Donald Knuth used pretty much that approach when writing TeX --- he started by writing out the sort of formatting/tagging which seemed appropriate, then theorizing about the sort of programming which would be appropriate for markup (hence macros), then worked on the implementation.
I've been trying a similar thing for my own effort to create a library for modeling G-code in OpenSCAD --- hopefully with the recent re-write in "pure" OpenPythonSCAD it will become something usable.
SICP is the best book to read as one's first book when studying computer science.
After many years of hobbyist programming (and consuming 'structured programming' books as well as languages from Pascal to Common LISP) we used Abelson & Sussmann at my undergraduate comp. sci. course, and it was eye-opening.
It demonstrates the simplicity, beauty and interactivity of Scheme while teaching you that computer science is the layering of different kinds of abstractions (from procedural abstraction and data abstraction, over defining your own (domain specific) language and implementing a compiler for it to defining new hardware in software). All of it seems so effortless, how only true masters can make things look like.
Make sure you buy the second edition, not the first or more recent ones, however (which use Python instead of Scheme - ugh).
I really wanted to like SICP but Lisp throws me off. I love Haskell and Standard ML however! Did others have a similar experience? Might be interesting to read a book similar in spirit to SICP but using a different language as a vehicle (No, I don't want to do SICP in JavaScript).
You might be interested in a 1987 article titled "A Critique of Abelson and Sussman or Why Calculating is Better than Scheming" (https://dl.acm.org/doi/10.1145/24697.24706), where the author advocates the use of KRC or Miranda as alternatives to Scheme. I don't know much about KRC, but Miranda is a statically-typed functional programming language that influenced Haskell.
SICP isn't a book about Lisp, however it uses some of Lisp's unique properties to demonstrate important concepts that other languages can't easily replicate. A book that's meant to be similar to SICP that doesn't use Scheme or Lisp would not be anything like SICP, or at least not teach the same things. Haskell and ML are in my experience much harder to understand than Scheme, so I'm wondering what your difficulty is?
I don't have a difficulty writing Lisp. It's just not my style. I don't like S-expressions. I understand this is not too big an issue and it's very likely I'll get more comfortable with it given enough time. It hasn't happened yet.
Have you looked at it? It's an abomination. The point of SICP isn't Scheme or the syntax of Scheme, but what it represents. Whoever made the Javascript rewrite didn't understand that. You can't write a metacircular interpreter in Javascript, because Javascript is not homoiconic.
I'm pretty sure someone wrote a very basic, very literal scheme to JavaScript transpiler and just ran the book's code through it. The results look nothing like what any normal person would write.
You can't write a metacircular interpreter in Javascript, because Javascript is not homoiconic.
Is that a downside? I never wrote or used metacurcular interpreter in my life and still don’t know why I had to read about it. Is it an interesting implementation technique of lisp? Yes. Does anyone really need that?
You can rip off that part and everything that follows and that will be enough for a regular programmer. No one itt needs to know how to design metacircular interpreter on register machines.
It allows the book to skip things like lexing and parsing and instead go straight to its main course: transforming, executing and compiling syntax trees.
I really wanted to like SICP and I probably would have if I read it 15 years ago. I started reading it last month and I found it to be too broad. It covers too much interesting mathematical principles and then jumps to the next one right when it starts to get interesting. In other words, it's too shallow.
It probably doesn't help that I've seen many courses/documents that are (in hindsight) derivatives from SICP, so I have the nagging thought "not this again" when a topic is introduced in SICP.
It's written for engineers, they already know the math, but they don't know how to design and implement virtual machines, objects, compilers and whatnot that it shows how to do.
I can identify with that - Lisp throws me off (because I’m not smart enough). But I ended up forcing myself to work through it and learned a tremendous amount because I’m not smart enough to work with a lisp. It felt like I spent so much time just reading through the code that I ended up learning more than I would in a language I’m comfortable with.
There is a Python version of SICP. I have never worked through it or even given it more than a cursory scan so this is not an endorsement more just a link to prove it exists:
"Functional Programming in Scala" aka "Red Book of Scala" is a the one that IMO teaches to think the same way as SICP, while using a typed language. The books stand next to each other on my bookshelf, definitely worth reading.
I'm not comfortable writing S-expressions. It feels very weird to me. I understand this is not a major issue. I'm hoping I stick with the book the next time I pick it up!
You should try out structured editing. For example paredit in emacs. And you really only need to understand slurp and barf. It really makes writing S-expressions no longer a chore. No more need to manually count parentheses. The parentheses are balanced by construction.
yes, i like ML (well.. ocaml) and bounced off SICP for the same reason. It was actually SICM that made me come back and stick with it, the ideas were just too interesting (whereas for SICP it was a lot of ideas I was already familiar with)
My second reading made me dig the footnotes and references, and there's a big world of beauty out there too. IIRC there's a paper where Sussman and some team made a custom design programmable processor to compute celestial bodies properties (trajectories). Mind bending as usual.
SICP helped me understand early on that there were many models of programming, even though I'd learned a limited number in my undergraduate. It was one of the books that helped me feel equipped to read the docs of any language, library or framework and have some notion of how to orient myself.
One of the best programming classes I had in college was a comparative languages course where multiple languages were covered, each in two week or so blocks.
Next to SICP, I like the entire "The Little *" series as reading twice (or more) material. And Types and Programming languages. For applicable (in what I do anyway) CS. But not only reading though; implementing as well; I need to repeat these things otherwise I forget parts.
I myself, but probably because I knew and respect the guy, I reread the works of Dijkstra ever so often; books + papers. Not really applicable anymore, but good for the brain and he was a good writer (imho).
I've been programming for 25 years and have owned the book for about 10 years. I just recently started to work through it and started with Dr. Racket.
There are things to love about Dr. Racket: hovering over a variable and visually seeing its connections to other places in the code is really cool. But ultimately I was a bit frustrated that it wasn't vs code.
So I stood up an configuration that let me use vs code (cursor actually) to work through the exercises. The LLM integration into cursor is cool as you can give it your code and whatever narrative you wrote and ask for feedback.
I am a tiny way through the exercises but having turned my code, the responses that I write, and the feedback that I get from the LLM into a static site.
It's been a fun way to spend a little time. For sure, I'm not getting the full benefit of working through SICP just with my own thoughts (without the aid of an LLM), but it's neat to see how you can integrate an LLM into the exercise.
this is not the Python version of SICP. It's a different book inspired by SICP. There's no "picture language" in chapter 2, and there's no "metacircular evaluator" and "register machine" in chapter 5.
Curious to hear folks opinion on the newer Software Design for Flexibility: How to Avoid Programming Yourself into a Corner (https://www.amazon.com/gp/aw/d/0262045494)?
If you want to get it elsewhere, the full info is:
Structure and interpretation of computer programs by Hal Abelson and Jerry Sussman (MIT Press. 1984. ISBN 0-262-01077-1).
I think this was mentioned in a Paul Graham essay from many years ago, but half the battle in tackling a new area of study is finding out what the best books (or papers) are. There aren’t that many, and yet it’s hard to know what they are if you aren’t already well-versed in the field.
Just so we're clear, this is a "beginner programming book" that has you create a scheme interpreter, then a register machine simulator, then a compiler out of your interpreter that will then have its compiled code run on the register machine simulator, by the final chapter.
This is probably the part where you'd step up and post a link to your repo with solutions to the exercises to back up your talk, but generally I only see this sort of casual dismissal from people who haven't actually worked through the book.
Concrete Abstractions, Schematics of Computation and others from the era (also using Scheme) covered similar ground (and went far further!) SICP is denser and sticks to theory forgoing databases, operating systems and actually implementing scheme in assembly.
I took cs61a at Berkeley as my very first computer science class I couldn't program I never tried to so scheme was my first language.
My ta told me that everybody should take the class twice when you first come in and when you're graduating.
When you first take it especially if you know other languages like C at the time you don't get the full depth of the problems you're given a great introduction and you think you understand everything but you don't realize the depth of complexity. Message passing the metacircular evaluator, continuations as the basis of all flow control, etc
You think they are neat tricks that you understand the curriculum because you can do the homework you don't understand how those neat tricks are really the basis of everything else you'll do.
When you're graduating you've had time to go through all your classes you realize just how foundation was principles are and you get so much more out of the book.
Well I didn't take the class a second time I need help grade and TA for a couple semesters.
I work as a quant developer and in trading now and even though my field has nothing to do with that I still think it's the basis of me as a developer.
The only computer science book worth reading twice, as suggested in discussions from 2010, is "Structure and Interpretation of Computer Programs" (SICP) by Hal Abelson and Jerry Sussman. This book is known for its deep insights into programming and its foundational concepts in computer science.
> The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology — the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects
Ironic, given the increasing use of functional programming in domains where old-fashioned imperative/OO programming used to reign alone.
I disagree since the book is using a functional programming language to advance the idea that CS is about procedural epistemology as opposed to the declarative stance of maths.
The idea that a 'procedural programming paradigm' exists in contrast with a 'functional programming paradigm' is blogspeak imho.
My understanding agrees with namaria's. I'm inclined to think that, in the passage you provide, `imperative' means `pertaining to processes' (where processes are those things described by procedures; or, perhaps better put, the meanings of procedures).
I’m slowly making my way through it a second time and thoroughly enjoying it. The first time through it seemed quite abstract, albeit only because of my completely lack of real world programming. The second time through it a revelation as I now have a strong base of experience through which to understand it (experience which it also, informs!).
I am using Elixir’s Livebook to take notes and complete the exercises. It is very helpful to have a live notebook tool while reading it!
I've read DDIA twice and I plan to read it again when the new edition comes out. And I will probably read it every couple of years again too. I can't really think of any other book I feel so strongly about personally.
- Godel, Escher, Bach - Hoffstadter
- The Soul of a New Machine - Kidder
- The Emperor's New Mind - Penrose
- The Connection Machine - Hillis
- Algorithmics - Harel
I’m working through it now, for someone with a computer engineering, EE or math background I think this is a great resource to get started with CS fundamentals.
Hot take: SICP and SD4F "considered harmful (without counterpoint)"*.
Why? The modus operandi of problem solving in these books is object oriented programming masquerading as functional programming, and it is presented as a _neutral_ beginner book. It is _not neutral_. This is a very opinionated approach to programming.
To be fair, I do not believe the authors intended for this style of programming to be taken as gospel, but it is often presented _without counterpoint_.
The most powerful technique introduced -- implementing complex behavior via extensible polymorphic generics -- is virtually unmaintainable without a compiler-supported static type checker. You would know that if you ever tried to implement the code yourself in a dynamic language of your choice.
The ramifications of these choices can be felt far and wide and are largely unquestioned.
Ironically, they make code hard to understand, hard to extend, and hard to maintain. I need to reiterate, I do not believe the intention of the authors was to suggest these ideas should be used beyond a pedagogical setting, but they often are.
As a specific critique to SD4F, which states as a goal making code more resilient by emulating biology, I would point to Leslie Lamport's talk on logic vs biology[1].
I would add that I think SICP would be fine if it were taught in tandem with Paradigms of Artificial Intelligence Programming by Peter Norvig[2]. PAIP offers a completely different approach to solving problems, also using lisp. This approach is much closer to constructing a language to model a problem and then solving the problem symbolically using the language created. Areas that use OO techniques, such as the chapter in CLOS, are clearly marked as such.
In other words, I say "SICP considered harmful" because thrusting it upon an eager newcomer as a trusted neutral guide to beginner coding (without offering any counterpoint) could set them back by a decade, filling their head with "functional object oriented programming" concepts that don't translate well to industry or CS.
[*]: I say this as someone who has thoroughly studied both books,
implemented the code, taken Dave Beazely courses to have the information spoon fed to me (dabeaz is awesome btw, take all his stuff) and used the techniques in production code bases.
> In other words, I say "SICP considered harmful" because thrusting it upon an eager newcomer as a trusted neutral guide to beginner coding (without offering any counterpoint) could set them back by a decade, filling their head with "functional object oriented programming" concepts that don't translate well to industry or CS.
I'd counter that by saying it would set them forward by a decade (compared to people who don't know these techniques). Knowing advanced techniques doesn't mean trying to shoehorn them into every run of the mill problem you encounter in the industry. But if you encounter a gnarly problem where some advanced techniques will help you out, you'll sure be glad you learnt them.
I would say that after 20+ years career OO plus functional elements have been a good default. Maybe I missed opportunities where symbolic computations were important and limited myself with languages like C++ that prefer OO/functional approaches... I can't go back and try something else to know =)
I might agree with your hot take in sense that leaving choice is important though.
This is great, but it’s not what I get paid for. I’ve yet to work at a place where I thought, “If only I had read SICP, things would be easier.”
I work with distributed systems, writing business logic and dealing with infrastructure concerns. For me, learning about databases, quirks of distributed systems, and patterns for building fault-tolerant services is more important than reading the nth book on structuring programs, deciding which algorithm to use, or figuring out whether my algorithm has O(1) or O(n) complexity.
This doesn’t mean CS fundamentals aren’t important—they are—but I work in a different space. I’d get more value out of reading Designing Data-Intensive Applications than SICP. If I were in the business of building frameworks or databases, I’d probably be the target audience.
There are many layers and dimensions. As pointed out that these days systems design, integration and how to interface is more important (and in fact in older days as well not sure why it becomes these days as system analysis is a job higher than programmer usually, at least 3-4 decades ago).
We need someone knowing this, just like we need someone to run the nuclear plant we use. But we do not need much those but we need more how to use electricity. Hence unlike another post physics are not the key even if it is more foundational.
For personal growth, it might be still though.
But frankly lisp is such a non-multi-system language, it has a hard time to deal with external world by its nature. It can be done as lisp is really the god level programming language. But as said it is NOT used by the gods for a reason.
We need find a system level language to express ourselves so that we can stand in giants. We need giants but no need to be one.
I picked up SICP expecting to read something really interesting or profound with the way it's been hyped up over the years however it's more of a how-to manual for working with Scheme/LISP and frankly that didn't interest me. Unfortunately most people have come to accept that LISP isn't a particularly effective way of programming even if some people get really excited by the idea of mutable and interchangeable data and code it's just not as powerful as they make it out to be and the obfuscation of program flow and execution and the lack of separation/delineation of data and code proves to be a hinderance more often than it is helpful. This doesn't discount LISP's contribution to computer science historically and how it's influenced modern day language design over the years, just that in my opinion LISP/SCHEME is more of a historical curiosity than a modern day guide to effective programming. (And certainly one that has no place as the introductory class at MIT). Anyway I've said something negative about SICP so prepare for this to be downvoted to the bottom :)
> In fact, I’d go further and say that it’s the only computer science book of that age that I’d happily and usefully read again without it being just for historical interest: the content has barely aged at all. That’s not all that unusual for mathematics books, but it’s almost unheard of in computer science, where the ideas move so quickly and where much of what’s written about is ephemeral rather than foundational.
I recall that when MIT stopped teaching with SICP, one of the main claims was that programming now is often not about thinking abstractions through from first principles, and creating some isolated gem of composing definitions. Instead, we interact with and rely on a rich ecosystem of libraries and tools which often have individual quirks and discordant assumptions, and engineering then takes on a flavor of discovering and exploring the properties and limitations of those technologies.
I think now, (some) people also are at the point of not even directly learning about the limitations and capability of each tool in their toolbox, but leaning heavily on generative tools to suggest low-level tactics. I think this will lead to an even messier future, where library code which works on (possibly generated) unit tests will bear some fragile assumption which was never even realized in the head of the engineer that prompted for it, and will not only fail but will be incorporated in training data and generated in the future.
I recall that when MIT stopped teaching with SICP, one of the main claims was that programming now is often not about thinking abstractions through from first principles, and creating some isolated gem of composing definitions.
Which is a category mistake that they actually address in the lectures. SICP is not a programming course, it’s a computer science course. Computer science is not about computers, let alone programming, just as geometry is not about surveying instruments and astronomy is not about telescopes.
When they stopped teaching SICP — in response to the pressure to teach more modern tools — they abandoned their scientific principles to satisfy commercial concerns. They stopped teaching computer science and became a vocational school for the tech industry.
I'm fine with the claim that CS is not about computers as astronomy is not about telescopes, but there are pure CS courses that don't cover programming and do cover automata, turing machines, computability, complexity etc and don't cover programming. SICP is about programs in a running language rather than abstract idealized computations, and is centered around reading and writing programs as examples. I think its success stems from its grounding in exhibiting such examples that ordinarily would not become accessible to students so quickly.
If you watch the lecture videos they make it a point of emphasis that they’re not interested in teaching LISP and that the course is not a LISP course. To show running programs they had to select some programming language. This was a regrettable concession because some students will inevitably think the course is about that language and miss the general principles they’re trying to teach:
1. Primitive operations
2. Means of combination
3. Means of abstraction
These are the three main features of programming languages, according to Sussman and Abelson. They wanted us to stop bikeshedding over the superficial details and just look at a tool for these 3 features, then be able to implement the algorithms and data structures we already know. This is how you become a wizard who can cast magic spells.
> SICP is not a programming course, it’s a computer science course.
I don't see what you mean by this at all. Furthermore this doesn't strike me as a useful distinction when a) it doesn't cover most topics labeled by consensus as "computer science" and b) it very clearly does teach a great deal about programming.
Why not say it teaches computer science and programming skills? Why do these have to be exclusive? There's obviously a great deal of overlap in general.
The goal of the course is not to teach programming skills, it's to teach computer science. The difference is explained quite thoroughly in the lectures. One might even say that answering the question "what is computer science?" is one of the core goals of the course and a major part of the philosophy of the professors who created the course.
The argument being made by the comparisons to geometry and astronomy is that in any discipline there is a difference between means and ends: what you are attempting to achieve is distinct from the tools you're using to achieve it. Furthermore, it's a mistake to believe that the discipline is all about the tools. No, the tools are the means, not the end.
> The goal of the course is not to teach programming skills, it's to teach computer science.
Who cares what the goal is? It teaches programming skills too. The intent is irrelevant and for the most part so too is the distinction (outside the american education system, anyway).
> Furthermore, it's a mistake to believe that the discipline is all about the tools.
Who outside the american education gives a damn about "the discipline", if that refers to anything meaningful outside the american education system in the first place? It's arbitrary and has no purpose or benefit aside from organizing the education system. This is a course that miraculously, against all odds, manages to teach useful skills in addition to jargon patterns of thought. Why not celebrate this?
Anyway, programming is a useful pedagogical tool for teaching CS. CS is a useful pedagogical tool for teaching programming. To brag about not teaching one is just hobbling your own insight into the value you provide students.
I myself have a CS degree from a prestigious institution and largely enjoyed my education. But this attitude you alude to is just jerking off for the sake of jerking off. Particularly in the case of SICP.
The quote was about "programming by poking" which I take as highly relevant to actual distributed software. It meant (1) systems are more built by integrating many components, and (2) for many reasons, the components are not understood by the integrator and (3) they must resort to experimentation to validate how things actually work.
Unless you have a TLA+ model of all your components and how they interact, I would argue you don't understand your distributed system either, for all inputs.
You don't think they can address both needs? What are non-MIT schools teaching when they teach CS if it's not SICP? Is everyone else just a vocational school?
SICP isn’t the only way to teach CS, obviously, but I’ll be honest with you: some schools aren’t even trying. They just offer Java and Python programming courses and call it a day.
I've witnessed how abandoning first principles undermines the evolution of a system. If our mental model of a system is not formalized into first principles (i.e. a high-level specification), then successive generations of engineers will have to re-learn those principles through trial-and-error. They'll introduce mutations and dependencies between the mutations-- and when they leave, the next generation of maintainers will repeat the process. Generations of mutations eventually create a brittle, calcified creature of a system which people fear to touch with a ten foot poll.
I imagine people who were taught SICP would be more respectful, if not inclined, towards a formal articulation of a system's principles.
I considered reading SICP recently but this changed my mind:
> It's old and feels old. originally in scheme, they recently re released the book in JavaScript which is more approachable to today's audiences and there are still good things in there about encapsulation and building dsls. ymmv. Though the language and programming design concepts hold up, we're playing at higher levels of abstraction on more powerful machines and consequently the examples sometimes seem too tiny and simple.
I had studied economics in a similar way, but learning slightly old/outdated ideas demotivated me - I was much more interested in learning what works and what's considered the best way to do things, not what had been considered a good idea at some point in the past.
I don't want to be a downer on SICP (especially since I haven't even read it), but I hope this info might help others (or elicit a strong refutation).
Scheme as basically an implementation of the untyped lambda calculus will eternally be a good frame work to think about the problems of computation in.
In the more practical area Racket (the most modern Scheme) has basically any practical functionality you would want, while amazingly remaining a platform for an incredible amount of experimentation in computation and programming language theory.
But SICP is a book that is for people interested in the study of computation what programming languages can be. If you're worried about getting a job in software it won't be all that useful, but it will remain a classic for anyone interested in engaging in creating the future of software.
It’s interesting, SICP and other many other “classic” texts talk about designing programs, but these days I think the much more important skill is designing systems.
I don’t know if distributed systems is consider part of “Computer Science” but it is a much more common problem that I see needs to be solved.
I try to write systems in the simplest way possible and then use observability tools to figure out where the design is deficient and then maybe I will pull out a data structure or some other “computer sciency” thing to solve that problem. It turns out that big O notation and runtime complexity doesn’t matter the majority of the time and you can solve most problems with arrays and fast CPUs. And even when you have runtime problems you should profile the program to find the hot spots.
What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.
The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.
Maybe I am just biased because I have a computer/electrical engineering background.
> but these days I think the much more important skill is designing systems.
It is hard to design systems if you don't have the perspective of implementing them. Yes, you move up the value chain to designing things, no, but no, you don't get to skip gaining experience lower down the value chain.
> What computer science doesn’t teach you is how memory caching works in CPUs.
That was literally my first quarter in my CS undergrad 30 years ago, the old Hennessy and Patterson book, which I believe is still used today. Are things so different now?
> The much more common problems I have is how to deal with fault tolerance, correctness in distributed locks and queues, and system scalability.
All of that was covered in my CS undergrad, I wasn't even in a fancy computer engineering/EE background.
I think CS 30 years ago was closer to computer engineering today.
At my uni 10 years ago the CS program didn’t touch anything related to hardware, hell the CS program didn’t even need to take multivariable calculus. In my computer engineering program we covered solid state physics, electromagnetism, digital electronics design, digital signals processing, CPU architecture, compiler design, OS design, algorithms, software engineering, distributed systems design.
The computer engineering program took you from solid state physics and transistor design to PAXOS.
The CS program was much more focused on logic proofs and more formalism and they never touched anything hardware adjacent.
I realize this is different between programs, but from what I read and hear many CS programs these days start at Java and never go down abstraction levels.
I do agree with you that learning the fundamentals is important, but I would argue that a SICP type course is not fundamental — physics is fundamental. And once you learn how we use physics to build CPUs you learn that fancy algorithms and complex solutions are not necessary most of the time given how fast computers are today. If you can get your CPU pipelined properly with high cache hits, branch prediction hits, prefetch hits, and SIMD you can easily brute force many problems.
And for those 10% of problems which cannot be brute forced, 90% of those problems can be solved with profiling and memoization, and for the 10% of those problems you cannot solve with memoization you can solve 90% of them with b-trees.
A top tier CS program is going to make you learn computer architecture along side automata and proofs. MIT went the extra mile with the SICP, it was honestly a hole I didn’t have access to in my top tier program, but I only realized this because I studied PL in grad school. You should go through it if you haven’t, I think it would have made my ugrad experience better and I definitely benefited from a great well rounded curriculum already (UW CSE is still no slouch, but it isn’t MIT!).
If you are into physics and mechanics, then you have to check the SICM (SICP’s less famous cousin) out as well. Again, MIT went the extra mile with that as well.
The CS degree at my school was pretty much just Java. Mostly UIs with java. And applets lol.
The only kids who learned anything else learned C++ so they could get jobs with DOD contractors
Computers today are slower than they have ever been. And tomorrow's computers are going to be even slower.
In many applications, the amount of data grows at least as quickly as computer performance. If the time complexity of an algorithm is superlinear, today's computer needs more time to run it with today's data than yesterday's computer did with yesterday's data. Algorithms that used to be practical get more and more expensive to run, until they eventually become impractical.
The more data you have, the more you have to think about algorithms. Brute-forcing can be expensive in terms of compute costs and wall-clock time, while low-level optimizations can take a lot of developer time.
if we're talking about relative human-experienced performance, the slowest computers i ever owned were in the early-mid 00s. they sped up as multicore and ssd's entered the picture and plateaued about ten years ago ime.
The most striking performance observation of my experience was that Apple took a system, OpenStep 4.2 which ran okay on a 33 MHz 68040 (and acceptably on my 25MHz Cube) and made it run only a little bit better on a 400MHz G3 as the Mac OS X Public Beta.
The difference of course was anti-aliasing, and much greater bit depth, and running multiple programming environments/toolkits (Carbon and Java).
We (at a public research university in the US) designed a rudimentary CPU, wrote mips assembly, and understood computer architecture for our CS degree. I graduated 6 years ago
Edit: we also did formal methods and proofs as part of core curriculum
> the old Hennessy and Patterson book, which I believe is still used today.
Do you mean "Computer Organization and Design - The Hardware / Software Interface" or "Computer Architecture: A Quantitative Approach"? Thanks.
The latter, I’m pretty sure.
> but these days I think the much more important skill is designing systems.
Isn’t t it what SICP is all about?!
Concur. 15 years ago, as sophomores we built a functional virtual cpu from the transistor level up. As I recall that was about 4 weeks, for the rest of the course, we extended the CPU with cache, extended opcode support, bus expansion, implemented a lexer/parser and ended writing our own assembly subset implementation and using that to write programs.
As for the OP's contention that computer science doesn't teach you to look for higher level things like cache thrashing, I wholeheartedly dissent with that supposition.
I recall at least 3 courses where substantial coursework was devoted to hacking, stack smashing, reverse compiling, profiling/introspection, kernel modification, so much beyond 'dynamic typing is an OO antipattern' stuff that gets IMO erroneously conflated with CS degrees.
Maybe these shit schools exist, but in a top 20 program you will definitely learn cache pitfalls.
Well CS and software dev in trenches moved a bit.
There are still jobs where people write frameworks, database engines or version control tools. Those jobs require heavy CS and algorithms, data structures day to day. But there are less of those jobs nowadays as no one is implementing db engine for their app they just use Postgres.
Other jobs that is vast majority is dealing with implementing business logic. Using database with understanding how it works in details is of course going to produce better outcomes. Yet one still can produce great amount of working software without knowing how indexes are stored on disk.
Also a lot of CS graduates fell into a trap where they think their job is to write a framework - where in reality they should just use frameworks and implement business logic- while using CS background to fully understand frameworks already existing.
> while using CS background to fully understand frameworks already existing.
Most frameworks today are so complicated that you typically cannot understand them fully, and even understanding them somewhat partially is more than a full-time job.
I wish someone told me this back when I was trying to get a programming job as a self taught programmer. I would do things like try to build a simple React clone thinking it would help me overcome imposter syndrome to fully understand things from the base up, but it was pretty futile because no one really has time to wrap their head around something that big unless they are paid full time to do it.
I have a saying that program's complexity is always exactly equal to the human-intelligible complexity + 1.
If not, the developer would add one more feature. It is due to the entirely human-made aspect of this discipline.
i did build a react clone in a long weekend. but I built the first 90% that takes 10% of the time and not the last 10% that takes 90% of time.
You don't have to understand them fully to make use of them. One has to understand design patterns and underlying reasoning, understand context of the framework like for example if it is JS framework that it runs in browser to understand which parts are "because we are running in the browser" vs "that is just why framework implemented it" and if it is Typescript then how it blends into that mix.
Then for any details or unexpected behavior knowing where to look in documentation.
Last week, after reading Methodology is bullshit here, this was my first thought!
https://x.com/henri__OK/status/1854813243916882365
> Yet one still can produce great amount of working software without knowing how indexes are stored on disk.
I agree... up to a point. Most software will likely be replaced/obsolete before it even reaches a scale where indexes even matter (at all) given how fast the underlying hardware is at this point.
... but I don't think this is particularly relevant wrt. the "to CS or not CS" question. If a CS grad has been paying any attention they usually have a decent idea of what kinds of problems are intractable vs. problems that are tractable (but maybe expensive to compute) vs. easy. Also just general exposure to different ways to approach solving a problem (logic programming, pure functional, etc.) can be very valuable. There's just much that one couldn't come up with on their own if one weren't exposed to the ideas from the vast expanse of ideas that are known in CS. (And even a master's doesn't come close to scratching the surface of it all.)
That's true, though new technologies -- web browsers, mobile devices -- have necessitated some framework writing.
So yes, some people working at Microsoft, Apple and Google wrote those frameworks. But that's like a drop in the bucket.
+Meta ;)
>What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.
Traditionally, the field of databases is largely about solving algorithm problems in the scenario where you have much more data that can fit in memory. Data exists on disk as "pages", you have a fixed number of "page slots" in RAM. Moving pages from disk to RAM or RAM to disk is slow, so you want to do as little of that as you can. This makes trivial problems interesting -- e.g. there's no notion of a 'join' in classic computer science because it's too trivial to bother naming.
We're used to thinking of the study of algorithms as a sort of pure essence, but one could argue that algorithmic efficiency is only meaningful in a particular data and hardware context. That's part of what keeps our jobs interesting, I guess -- otherwise algorithm expertise wouldn't be as useful, since you could just apply libraries/cookbook solutions everywhere.
True. However I find that most junior and even experienced programmers struggle with tactical level coding. I'm really suffering with this right now because the small component I'm tasked with making a small change to is annoyingly stateful and deals with 2 abstractions at once. (It processes files and uses the file system and database to store it's state). I'm shocked how badly it has been thought out. I've spent days trying to avoid doing what has gone before, bits bolted on that make it even more difficult to understand. It really seems that pull request culture has just led to any old crap being approved because no one has the band width to think deeply about the actual code. Bring back in person code reviews !
Have you seen
"Software Design for Flexibility: How to Avoid Programming Yourself into a Corner" by Chris Hanson and Gerald Jay Sussman
It's from 2021.
https://archive.org/details/software-design-for-flexibility_...
I hadn't, that looks excellent.
Unfortunately, while I really want to love Software Design for Flexibility, it's clear that Hanson and Sussman haven't really solved (or even come close to solving) the problem they have identified in the book.
The introduction to that book is brilliant at identifying just how much room software has to grow (you can find similar talks from various Strange Loop sessions Sussman has done), and is really quite inspirational for anyone seriously thinking about the future of computing.
But the rest of the book fails to provide a coherent answer to the questions that are brought up in the intro. It shows off some neat functional programming tricks, but repeatedly fails to deliver on solving the (admittedly ambitious) challenges it provides for itself.
I'm still glad I have a copy, and have re-read the first half multiple times now, but sadly it's not the book it wants to be. To be fair though, that is because we haven't come close to understanding computation enough to solve those problems.
It's a very ambitious book that falls short of it's own ambitious.
A more humble book, with a more grounded approach might be:
_A Philosophy of Software Design_ by John Ousterhout (the guy behind Tcl)
https://www.goodreads.com/book/show/39996759-a-philosophy-of...
Written as the textbook for a software engineering course, it developed out of that course being taught multiple times _and_ all the code reviews which that entailed.
Previous discussions/mentions here which had a notable number of comments:
https://news.ycombinator.com/item?id=41017367
https://news.ycombinator.com/item?id=34733120
https://news.ycombinator.com/item?id=17779953
https://news.ycombinator.com/item?id=8055868
I re-wrote my current project in the course of reading it (I would read a chapter, then read through the code and where appropriate apply the relevant principle) and once I finish the current re-write (from OpenSCAD to Python) will be repeating that process to see if what I was supposed to have learned stuck/survived the re-write.
IMO it's not excellent. It's not like SICP, it's obtuse for no reason, I find it a hard slog. Flexibility is good but it seems to try to make every bit of your program flexibile and pluggable and you just need to do something eventually.
My opinion, I'd welcome others on the book; there was a small splash when it came out but not much discussion since.
I haven't read the book, but my experience is that the way to make things flexible is to make them simple as possible.
When I've used (or built) something that was built in the style like you're talking about, it's almost always wrong, and the extra complexity and stuff now makes it harder to do right. It's not surprising: unknown future requirements are unknown. Over building is trying to predict the future.
It's like someone building a shed and pouring a foundation that can work for a skyscraper. Except it turns out what we needed was a house that has a different footprint. Or maybe the skyscraper is twice the height and has a stop for the newly-built underneath. Now we have to break apart the foundation before we can even begin work on new stuff; it would have been less work if the original just used a foundation for a shed.
I think that is what the book does. Step by step introducing new requirements, by telling you about a situation, where the previous code would not be flexible enough. I am sure it doesn't state, that one should apply all of its ideas all the time, regardless of the project at hand.
Is there another book you'd recommend - more recent than SICP - for how to avoid programming yourself into a corner?
I don't know about books, but I think the best approach is functional programming in a dynamic language. That could be because I'm currently an Elixir fanboy, but I think Lisps, especially Scheme or Clojure, or a functional-restricted approach in JavaScript could do it as well. I agree with parent comment that it's better to keep things as simple as possible and make the changes when necessary vs. building in all the flexibility in the beginning.
Scouring SICP cannot imbue the student with mechanical sympathy any more than poring over analysis of Coltrane makes me a saxophonist.
Nevertheless. It must be done. Theory and practice.
Nicely said. The way I think about it: if we can’t write legible and adaptable functions, then we have no chance at making viable systems. All the same engineering skills are at play, just on a different scale.
I find books like SICP interesting and not very useful. I love reading them because I like this stuff, but I don’t get to apply their teachings in real world software. It’s a problem because naturally I want to spend my time reading these kind of books, but if I do that I would be jobless. I need to divide my time between reading pearls like SICP and boring Kafka/Postgres/Golang/K8s/AWS documentation.
I don’t find them useful in the sense of directly applying practical techniques in my day job, but I consider them somewhat necessary background reading to get into the right state of mind. You can very quickly tell when someone never acquired any academic knowledge in this area (or never played with functional languages or similar pastimes) - you can’t explain to those people why modifying global variables all over the place in a large program is a bad idea and other things like that. They just nod along skeptically and then somehow keep stumbling into the same kind of mess over and over.
You kind of defeat your own argument. You say it's important to learn "academic knowledge", but then acknowledge the organization will not value your knowledge.
I do agree with you though.
Well in my experience good organizations do recognize that the better design means lower costs in the long run, and people who don’t get that tend to not get promoted. Communicating this effectively up and down the chain is a whole different art in itself though.
One of the problems I've seen is that when new learners and self-taught individuals ask for advice, a lot of software engineers give recommendations based on what they wish their job was or how they would like to imagine themselves.
This is a real problem that people with experience put on learners. If you asked them how they learned it, they'd tell you an entirely different story about starting small, practicing often, trying many ideas, having a strong motivation and some occasional guidance. But they tell others to follow a rigorously defined path which creates the opposite mindset that a learner needs.
The first reason why I really loved SICP is that it is based on Scheme, a language with powerful primitives. I came from a self-taught world of PL/1, Algol, C, then later C++, Java etc. None of them had closures, hygienic macros, anonymous functions, functional programming, call/cc, and of course, "amb", the non-deterministic choice operator. At an even more basic level, SICP taught me that a lot of non-trivial code can be written with just sequences and maps, with good enough efficiency!
Because SICP's starting point was so high, they could describe many concepts easily from the ground up, from object oriented programming, backtracking, constraint programming and non-determinism.
This taught me a number of techniques to apply in real-life, because I could readily identify the missing building blocks in the language or system I was given to work with. For example, I was able to build a lightweight threads system in Java quite readily because I knew that the missing piece was a continuations feature in Java.
See https://github.com/kilim/kilim
You're in luck. Part 5 of the book is about building a virtual machine to run lisp simalated at the register lelvel: https://mitp-content-server.mit.edu/books/content/sectbyfn/b...
Writing a network between n such machines is left as an exercise to the reader.
> I think the much more important skill is designing systems
Systems engineering is a seperate discipline in engineering fields, same applies to CS I would think, specifically if you look at what most developers do as computer engineering it would only make sense that there is forward progression within the discipline that builds on the foundations tought in most CS courses.
Regarding CS not teaching caching, well any course writer in a CS program that doesn't at least touch on that should feel like they failed their students, it is really something fundamental that should be tought pretty soon.
I feel like profiling should be something that is tought somewhere in a CS program, even a half semester course can dramatically improve peoples understanding of what the machine is actually doing.
On that note, even if you never intend to use the language, watch some C++ conference talks, almost every year at every conference thwre is a talk discussing performance.
> these days I think the much more important skill is designing systems
That's true, but that doesn't mean that there is no value in having an understanding of how established technology works under the hood.
> What computer science doesn’t teach you is how memory caching works in CPUs.
That is also a very good point. There is a lot of daylight between the lambda calculus and real systems.
> big O notation and runtime complexity doesn’t matter the majority of the time and you can solve most problems with arrays
I have the exact opposite experience.
Software comes out best if you always ensure to use an approach with sensible runtime complexity, and only make trade-offs towards cache-friendly-worse-O implementations where you benchmarked thoroughly.
Most cases where I encounter mega slow programs are because somebody put in something quadratic instead of using a simple, standard O(n logn) solution.
Check out https://www.tumblr.com/accidentallyquadratic for many examples.
The right book for the right problem. SICP isn't meant to teach you how to tackle fault-tolerance in a complex distributed system. Here is a textbook that talks about distributed systems (van Steen and Tannenbaum):
https://www.amazon.ca/Distributed-Systems-Maarten-van-Steen/...
You can also get a free PDF version of that textbook here https://www.distributed-systems.net/index.php/books/ds4/ (you only need to provide an email)
Yes, I have the distributed system book from van Steen :)
> What computer science doesn’t teach you is how memory caching works in CPUs. Your fancy graph algorithm may have good runtime complexity but it completely hoses the CPU cache and you may have been able to go faster with an array with good cache usage.
Computer architecture and organization should teach this, no?
> What computer science doesn’t teach you is how memory caching works in CPUs.
Yes it can, and there are tons of papers about data structures to use in various scenarios to handle not just L1, L2, L3, but also NUMA. Sure, this isn’t in SICP, but claiming CS as a field completely ignores how memory works is incorrect.
There are still innumerable people writing standalone programs, single-purpose embedded systems, independent components and libraries, etc
The industry has expanded to include a lot of large-scale distributed cloud projects (often where we might have expected mainframes and cobol before), with many of today's largest employers doing most of their work there, but none of that other stuff really went away. It's still being done every day.
You need a book for what you're doing, and not every book is going to be that. Apparently, SICP is not it. I possess and have read many books, and only some small number of them are applicable to the projects I'm working on at any time.
They don't compete with each other, they complement each other.
If you knew how to design programs you could run it all on a single box and wouldn’t have to design “systems.”
I’m being slightly facetious, but only slightly. If you really think everything is solvable with arrays, you are not going to scale well and of course you’re going to need to throw a lot more hardware at the problem.
My argument is that 90% of problems can be solved with arrays, 5% of problems can be solved with memoization, 3% of problems can be solved with b-trees, and 2% of problems with other data structures.
It is good to know that solutions to the 2% exists, but what we should be focusing on is writing the simplest code possible which solves the problem and then only optimize afterwards using a profiler. God forbid you have to work on some codebase written by someone who believes they are the second coming of haskell with crazy recursion and backtracing, monads, red black trees, and a DSL on top of the whole thing.
You are right that many problems can be solved with a single box, but my argument is that you do not need fancy algorithms to solve problems on a single box. We should strive to use single boxes whenever possible to reduce complexity.
Computation is designed by humans to serve humans, we should make it as easy as possible for humans to understand. I’m probably going to start a flamewar here, but this is why simple solutions like UNIX and golang have prevailed in the past. Simple code is easy to understand and therefore it is easy to modify and reason about. Some people think simple means that you decompose programs into the smallest possible functional parts, but simple to me is a 500 line main function.
Since you emphasize designing systems over just programs, do you have any go-to resources or references ?
My CS degree covered hardware architecture, disk design, OS design, and also system design, alongside algorithms, data-structures, various programming language and paradigm, etc.
> I don’t know if distributed systems is consider part of “Computer Science"
It surely was part of my Informatics Engineering degree, with Tanenbaum book being one of the required reads.
I think abstraction layers exist at the system design level as well. Many of the things SICP teaches apply there as well.
Do you have any distributed systems design books to recommend?
Distributed Systems by Van Steen and Tanenbaum is very good and freely bailable:
https://www.distributed-systems.net/
I think the classic CLR text is great but, yeah, caches through quite a monkey wrench into naive big O analysis.
Still, I think the time investment to learn algos and data structures isn't too much of a burden.
[dead]
The article has a broken link for the free copy:
https://mitp-content-server.mit.edu/books/content/sectbyfn/b...
https://web.mit.edu/6.001/6.037/sicp.pdf
I hadn't seen a blessed PDF version until today. Circa 2001, only the HTML version was freely available, and someone converted it to TeXinfo: https://www.neilvandyke.org/sicp-texi/
If anyone wants to work through SICP today, you can run the code in MIT Scheme, or in DrRacket: https://www.neilvandyke.org/racket/sicp/
For anyone wishing to try: the maintainers of MIT Scheme no longer provide a .dmg but you can download and build the x86_64 version of MIT Scheme. The current release (v12.1) works on a Mac running Sequoia with Intel CPU or on Apple silicon via Rosetta. But the native code compiler (not necessary for SICP AFAIK) is a little broken. (Anecdotally it worked on macOS prior to Monterey, so maybe an Apple-supplied dependency changed. Haven't tracked down the issue.)
All of that is to say: if you do not need MIT Scheme and don't want to fuss with compiling it, then Racket might be a better way to go.
There isn't an active maintainer any more, I'm afraid. And Apple placed restrictions on modifying memory that contains instructions that prevented techniques the compiler relies upon for fast allocation of closures. There are ideas for workarounds, but they would require a lot of work.
I'm a huge fan of MIT Scheme, and have used it since 1984, but I would recommend using another implementation these days, especially on Mac.
See Scheme.org.
most package managers have it, including apt and brew, so most of the time no need to build your own
Good point! though my comment about the native code compiler being broken still applies to the brew-installed version
Just as a data point, I'd recommend going through it in Racket, which I believe has an explicit SICP mode. I went through it in GNU Guile and it was a pain because there were some minor syntactic differences between Guile and MIT Scheme.
The texinfo version was I believe the source for the really nice HTML5 version if you want to read it in a browser, but with nice formatting that the MIT original version: https://sarabander.github.io/sicp/
one thing to note is that the second chapter's "picture language" is not supported in MIT Scheme in 2024. There used to be a package but it's like 2 decades out of maintenance. In Dr. Racket however, there is a package specifically for working through those problems.
Dr Racket has SICP and HTDP as a teaching pack.
I haven’t seen it in the comments yet, but you can watch Abelson and Sussman teaching the material from this book from recorded lectures in 1986.
I still find their description of how to create and group abstractions in various layers to be useful personally and as a mentor. (In the videos, lesson 3A, 1:07:55)
https://m.youtube.com/playlist?list=PLE18841CABEA24090
The Kabbalah joke gets me every time.
Thanks for sharing this. It's crazy to see videos like this and think how these people had no idea what was coming in the future.
Just dropping in to say that The Elements of Programming Style is worth reading three times — and I have read it many more times than that, and benefitted from it. Here's my review (from 2010) if you're interested: https://reprog.wordpress.com/2010/03/06/programming-books-pa...
Oh and here I thought you were talking about Elements of Programming by Stepanov and McJones which tbh I'd give the same recommendation/review.
https://elementsofprogramming.com/
My favourite part of SICP and something that has stuck with me for years is the idea of "wishful programming". That is where you build something top-down by simply wishing you had the lower-level routines. Then, of course, you actually go and build those lower-level routines until you reach the bottom. I find this way of thinking works really well with test-driven development. Write a test against functionality you wish you had, then go and fulfill that wish. Most developers seem to build stuff bottom-up and then end up with something that isn't really what anyone wished for.
They do that because their wish is performance and naturalness.
You may accidentally wish something you don’t yet know the true nature of, and this will create a fragile mess at the bottom. It usually does, cause algorithmic nature of things is rarely intuitive. Starting from the bottom is like starting from quarks that you have rather than from “I want magic to exist”. Well it does not. You reach the bottom and there’s quarks instead of magicules and you’ve lost all context clues on the way which could help to convert between two physics.
Both approaches have their use, because sometimes you have to be bold with your wishes to solve a deep problem. But personally I prefer magic to be packed into the before-topmost layer. I.e. build from the bottom up, and then, just before the business logic, create a convenience magic layer that translates to/from business speak. It becomes adjustable and doesn’t induce a tangled mess all the way down.
So I think one of the things best avoided in life in general is extremity, in all its various guises. No single technique should be followed like scripture, but rather incorporated into ones toolkit and used where appropriate. Building top-down will get you where you want, but risks fragile underpinnings due to a lack of cross-cutting architectural guidance. But, on the other hand, bottom-up might get you the best foundations at each layer but ultimately deliver nothing of value to the users. In practice it's necessary to take a balanced approach and, of course, mistakes will be made and experience will become the guide. Like you I definitely employ the "meet in the middle" approach in practice, but what SICP taught me is how to think about starting at the top, that is, to build upon wishes.
Interestingly, Dr. Donald Knuth used pretty much that approach when writing TeX --- he started by writing out the sort of formatting/tagging which seemed appropriate, then theorizing about the sort of programming which would be appropriate for markup (hence macros), then worked on the implementation.
I've been trying a similar thing for my own effort to create a library for modeling G-code in OpenSCAD --- hopefully with the recent re-write in "pure" OpenPythonSCAD it will become something usable.
The Smalltalk world has great support for this, through coding in the debugger. You should try Pharo.
SICP is the best book to read as one's first book when studying computer science.
After many years of hobbyist programming (and consuming 'structured programming' books as well as languages from Pascal to Common LISP) we used Abelson & Sussmann at my undergraduate comp. sci. course, and it was eye-opening.
It demonstrates the simplicity, beauty and interactivity of Scheme while teaching you that computer science is the layering of different kinds of abstractions (from procedural abstraction and data abstraction, over defining your own (domain specific) language and implementing a compiler for it to defining new hardware in software). All of it seems so effortless, how only true masters can make things look like.
Make sure you buy the second edition, not the first or more recent ones, however (which use Python instead of Scheme - ugh).
I really wanted to like SICP but Lisp throws me off. I love Haskell and Standard ML however! Did others have a similar experience? Might be interesting to read a book similar in spirit to SICP but using a different language as a vehicle (No, I don't want to do SICP in JavaScript).
You might be interested in a 1987 article titled "A Critique of Abelson and Sussman or Why Calculating is Better than Scheming" (https://dl.acm.org/doi/10.1145/24697.24706), where the author advocates the use of KRC or Miranda as alternatives to Scheme. I don't know much about KRC, but Miranda is a statically-typed functional programming language that influenced Haskell.
SICP isn't a book about Lisp, however it uses some of Lisp's unique properties to demonstrate important concepts that other languages can't easily replicate. A book that's meant to be similar to SICP that doesn't use Scheme or Lisp would not be anything like SICP, or at least not teach the same things. Haskell and ML are in my experience much harder to understand than Scheme, so I'm wondering what your difficulty is?
I don't have a difficulty writing Lisp. It's just not my style. I don't like S-expressions. I understand this is not too big an issue and it's very likely I'll get more comfortable with it given enough time. It hasn't happened yet.
There's a SICP edition done in Javascript.
Have you looked at it? It's an abomination. The point of SICP isn't Scheme or the syntax of Scheme, but what it represents. Whoever made the Javascript rewrite didn't understand that. You can't write a metacircular interpreter in Javascript, because Javascript is not homoiconic.
I'm pretty sure someone wrote a very basic, very literal scheme to JavaScript transpiler and just ran the book's code through it. The results look nothing like what any normal person would write.
You can't write a metacircular interpreter in Javascript, because Javascript is not homoiconic.
Is that a downside? I never wrote or used metacurcular interpreter in my life and still don’t know why I had to read about it. Is it an interesting implementation technique of lisp? Yes. Does anyone really need that?
You can rip off that part and everything that follows and that will be enough for a regular programmer. No one itt needs to know how to design metacircular interpreter on register machines.
It allows the book to skip things like lexing and parsing and instead go straight to its main course: transforming, executing and compiling syntax trees.
I really wanted to like SICP and I probably would have if I read it 15 years ago. I started reading it last month and I found it to be too broad. It covers too much interesting mathematical principles and then jumps to the next one right when it starts to get interesting. In other words, it's too shallow.
It probably doesn't help that I've seen many courses/documents that are (in hindsight) derivatives from SICP, so I have the nagging thought "not this again" when a topic is introduced in SICP.
It's written for engineers, they already know the math, but they don't know how to design and implement virtual machines, objects, compilers and whatnot that it shows how to do.
I can identify with that - Lisp throws me off (because I’m not smart enough). But I ended up forcing myself to work through it and learned a tremendous amount because I’m not smart enough to work with a lisp. It felt like I spent so much time just reading through the code that I ended up learning more than I would in a language I’m comfortable with.
There is a Python version of SICP. I have never worked through it or even given it more than a cursory scan so this is not an endorsement more just a link to prove it exists:
https://wizardforcel.gitbooks.io/sicp-in-python/content/0.ht...
"Functional Programming in Scala" aka "Red Book of Scala" is a the one that IMO teaches to think the same way as SICP, while using a typed language. The books stand next to each other on my bookshelf, definitely worth reading.
I don't understand why Lisp throws you off. I only read SICP after I became proficient in Haskell and it is just fine.
I'm not comfortable writing S-expressions. It feels very weird to me. I understand this is not a major issue. I'm hoping I stick with the book the next time I pick it up!
You should try out structured editing. For example paredit in emacs. And you really only need to understand slurp and barf. It really makes writing S-expressions no longer a chore. No more need to manually count parentheses. The parentheses are balanced by construction.
"Concepts, Techniques, and Models of Computer Programming" by Peter Van Roy and Seif Haridi is my favourite.
i thought berkeley was using a modified version of the book using python a few years back
yes, i like ML (well.. ocaml) and bounced off SICP for the same reason. It was actually SICM that made me come back and stick with it, the ideas were just too interesting (whereas for SICP it was a lot of ideas I was already familiar with)
I think there is a Python version if that floats your boat
My second reading made me dig the footnotes and references, and there's a big world of beauty out there too. IIRC there's a paper where Sussman and some team made a custom design programmable processor to compute celestial bodies properties (trajectories). Mind bending as usual.
SICP helped me understand early on that there were many models of programming, even though I'd learned a limited number in my undergraduate. It was one of the books that helped me feel equipped to read the docs of any language, library or framework and have some notion of how to orient myself.
One of the best programming classes I had in college was a comparative languages course where multiple languages were covered, each in two week or so blocks.
Next to SICP, I like the entire "The Little *" series as reading twice (or more) material. And Types and Programming languages. For applicable (in what I do anyway) CS. But not only reading though; implementing as well; I need to repeat these things otherwise I forget parts.
I myself, but probably because I knew and respect the guy, I reread the works of Dijkstra ever so often; books + papers. Not really applicable anymore, but good for the brain and he was a good writer (imho).
I've been programming for 25 years and have owned the book for about 10 years. I just recently started to work through it and started with Dr. Racket.
There are things to love about Dr. Racket: hovering over a variable and visually seeing its connections to other places in the code is really cool. But ultimately I was a bit frustrated that it wasn't vs code.
So I stood up an configuration that let me use vs code (cursor actually) to work through the exercises. The LLM integration into cursor is cool as you can give it your code and whatever narrative you wrote and ask for feedback.
I am a tiny way through the exercises but having turned my code, the responses that I write, and the feedback that I get from the LLM into a static site.
It's been a fun way to spend a little time. For sure, I'm not getting the full benefit of working through SICP just with my own thoughts (without the aid of an LLM), but it's neat to see how you can integrate an LLM into the exercise.
Original version: https://mitp-content-server.mit.edu/books/content/sectbyfn/b...
Javascript version: https://sourceacademy.org/sicpjs/index
And the Python version: http://www.composingprograms.com/
this is not the Python version of SICP. It's a different book inspired by SICP. There's no "picture language" in chapter 2, and there's no "metacircular evaluator" and "register machine" in chapter 5.
It's hard to understand the point without these.
Curious to hear folks opinion on the newer Software Design for Flexibility: How to Avoid Programming Yourself into a Corner (https://www.amazon.com/gp/aw/d/0262045494)?
It’s a much, much denser successor to sicp. I hadn’t succeeded in self-studying with it despite strong lisp/scheme chops and strong affinity for sicp.
I have a copy. Found it fun, but not quite as mind shifting. I think I need to try it again, but I am curious how others feel.
i'd also recommend "Concrete Abstractions: An Introduction to Computer Science using Scheme" by Max Hailperin, Barbara Keiser, Karl Knight.
http://www.gustavus.edu/+max/concrete-abstractions.html
Spelling correction on the second author's last name: Kaiser
Thank you for the correction. pardon my typo.
I'll second that recommendation, it's a bit like a gentle mixture between SICP and TAOCP. A very enjoyable read.
I concur, I am learning from it now…
SICP is available for free: https://web.mit.edu/6.001/6.037/sicp.pdf
If you want to get it elsewhere, the full info is: Structure and interpretation of computer programs by Hal Abelson and Jerry Sussman (MIT Press. 1984. ISBN 0-262-01077-1).
Programming Pearls is another book that rereads well. It's also short, too, which makes rereading it possible.
There are some great books, and every book means something different to each person who reads it.
K&R influenced a generation of programmers.
Hennessy and Patterson influence a generation of architects.
etc. etc.
It's not just SICP.
But the greater point: a book can be meaningful, and we can always use more good ones.
I think this was mentioned in a Paul Graham essay from many years ago, but half the battle in tackling a new area of study is finding out what the best books (or papers) are. There aren’t that many, and yet it’s hard to know what they are if you aren’t already well-versed in the field.
I don’t quite get the cult status of SICP. I read it and it’s a fine beginner programming book, but nothing more.
Just so we're clear, this is a "beginner programming book" that has you create a scheme interpreter, then a register machine simulator, then a compiler out of your interpreter that will then have its compiled code run on the register machine simulator, by the final chapter.
This is probably the part where you'd step up and post a link to your repo with solutions to the exercises to back up your talk, but generally I only see this sort of casual dismissal from people who haven't actually worked through the book.
I commend your righteous indignation. Made me smile. Flame on!
One aspires to be a hater of such high caliber.
Concrete Abstractions, Schematics of Computation and others from the era (also using Scheme) covered similar ground (and went far further!) SICP is denser and sticks to theory forgoing databases, operating systems and actually implementing scheme in assembly.
I don’t understand this comment. If you master the material you know more than 90% of engineers in the field.
> If you master the material you know more than 90% of engineers in the field.
Telling someone that he/she is smarter than 90% of the people is not a praise. :-)
amen… just look at 90% of people at the DMV :-)
I took cs61a at Berkeley as my very first computer science class I couldn't program I never tried to so scheme was my first language.
My ta told me that everybody should take the class twice when you first come in and when you're graduating.
When you first take it especially if you know other languages like C at the time you don't get the full depth of the problems you're given a great introduction and you think you understand everything but you don't realize the depth of complexity. Message passing the metacircular evaluator, continuations as the basis of all flow control, etc
You think they are neat tricks that you understand the curriculum because you can do the homework you don't understand how those neat tricks are really the basis of everything else you'll do.
When you're graduating you've had time to go through all your classes you realize just how foundation was principles are and you get so much more out of the book.
Well I didn't take the class a second time I need help grade and TA for a couple semesters.
I work as a quant developer and in trading now and even though my field has nothing to do with that I still think it's the basis of me as a developer.
My same experience. For much of the rest of the cs curriculum I felt like we had already to some extent covered the main ideas in 61a with sicp.
The only computer science book worth reading twice, as suggested in discussions from 2010, is "Structure and Interpretation of Computer Programs" (SICP) by Hal Abelson and Jerry Sussman. This book is known for its deep insights into programming and its foundational concepts in computer science.
> The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology — the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects
Ironic, given the increasing use of functional programming in domains where old-fashioned imperative/OO programming used to reign alone.
I think in the context of the book 'procedural epistemology' encompasses all programming, not just what you'd call procedural programming.
Hmm, I don’t think so. Functional programming is definitely based on the “declarative point of view taken by classical mathematical subjects”.
I disagree since the book is using a functional programming language to advance the idea that CS is about procedural epistemology as opposed to the declarative stance of maths.
The idea that a 'procedural programming paradigm' exists in contrast with a 'functional programming paradigm' is blogspeak imho.
I see. I didn’t realize the book uses a functional language. Thank you for explaining.
My understanding agrees with namaria's. I'm inclined to think that, in the passage you provide, `imperative' means `pertaining to processes' (where processes are those things described by procedures; or, perhaps better put, the meanings of procedures).
I’m slowly making my way through it a second time and thoroughly enjoying it. The first time through it seemed quite abstract, albeit only because of my completely lack of real world programming. The second time through it a revelation as I now have a strong base of experience through which to understand it (experience which it also, informs!).
I am using Elixir’s Livebook to take notes and complete the exercises. It is very helpful to have a live notebook tool while reading it!
You're doing the exercises in Elixir and not Scheme then?
I've read DDIA twice and I plan to read it again when the new edition comes out. And I will probably read it every couple of years again too. I can't really think of any other book I feel so strongly about personally.
My twice reading list:
I’m working through it now, for someone with a computer engineering, EE or math background I think this is a great resource to get started with CS fundamentals.
Always fun to see one of my professors from the quite tiny, but awesome computer science department at St Andrews on HN!
I love SICP, but in industrial software I'd point to "The Mythical Man Month".
Hot take: SICP and SD4F "considered harmful (without counterpoint)"*.
Why? The modus operandi of problem solving in these books is object oriented programming masquerading as functional programming, and it is presented as a _neutral_ beginner book. It is _not neutral_. This is a very opinionated approach to programming.
To be fair, I do not believe the authors intended for this style of programming to be taken as gospel, but it is often presented _without counterpoint_.
The most powerful technique introduced -- implementing complex behavior via extensible polymorphic generics -- is virtually unmaintainable without a compiler-supported static type checker. You would know that if you ever tried to implement the code yourself in a dynamic language of your choice.
The ramifications of these choices can be felt far and wide and are largely unquestioned.
Ironically, they make code hard to understand, hard to extend, and hard to maintain. I need to reiterate, I do not believe the intention of the authors was to suggest these ideas should be used beyond a pedagogical setting, but they often are.
As a specific critique to SD4F, which states as a goal making code more resilient by emulating biology, I would point to Leslie Lamport's talk on logic vs biology[1].
I would add that I think SICP would be fine if it were taught in tandem with Paradigms of Artificial Intelligence Programming by Peter Norvig[2]. PAIP offers a completely different approach to solving problems, also using lisp. This approach is much closer to constructing a language to model a problem and then solving the problem symbolically using the language created. Areas that use OO techniques, such as the chapter in CLOS, are clearly marked as such.
In other words, I say "SICP considered harmful" because thrusting it upon an eager newcomer as a trusted neutral guide to beginner coding (without offering any counterpoint) could set them back by a decade, filling their head with "functional object oriented programming" concepts that don't translate well to industry or CS.
[*]: I say this as someone who has thoroughly studied both books, implemented the code, taken Dave Beazely courses to have the information spoon fed to me (dabeaz is awesome btw, take all his stuff) and used the techniques in production code bases.
[1]: https://lamport.azurewebsites.net/pubs/future-of-computing.p...
[2]: https://github.com/norvig/paip-lisp
> In other words, I say "SICP considered harmful" because thrusting it upon an eager newcomer as a trusted neutral guide to beginner coding (without offering any counterpoint) could set them back by a decade, filling their head with "functional object oriented programming" concepts that don't translate well to industry or CS.
I'd counter that by saying it would set them forward by a decade (compared to people who don't know these techniques). Knowing advanced techniques doesn't mean trying to shoehorn them into every run of the mill problem you encounter in the industry. But if you encounter a gnarly problem where some advanced techniques will help you out, you'll sure be glad you learnt them.
I would say that after 20+ years career OO plus functional elements have been a good default. Maybe I missed opportunities where symbolic computations were important and limited myself with languages like C++ that prefer OO/functional approaches... I can't go back and try something else to know =)
I might agree with your hot take in sense that leaving choice is important though.
Many classics are worth reading twice. Knuth, Tanenbaum, Stephens, PAIP, ...
This is great, but it’s not what I get paid for. I’ve yet to work at a place where I thought, “If only I had read SICP, things would be easier.”
I work with distributed systems, writing business logic and dealing with infrastructure concerns. For me, learning about databases, quirks of distributed systems, and patterns for building fault-tolerant services is more important than reading the nth book on structuring programs, deciding which algorithm to use, or figuring out whether my algorithm has O(1) or O(n) complexity.
This doesn’t mean CS fundamentals aren’t important—they are—but I work in a different space. I’d get more value out of reading Designing Data-Intensive Applications than SICP. If I were in the business of building frameworks or databases, I’d probably be the target audience.
There are many layers and dimensions. As pointed out that these days systems design, integration and how to interface is more important (and in fact in older days as well not sure why it becomes these days as system analysis is a job higher than programmer usually, at least 3-4 decades ago).
We need someone knowing this, just like we need someone to run the nuclear plant we use. But we do not need much those but we need more how to use electricity. Hence unlike another post physics are not the key even if it is more foundational.
For personal growth, it might be still though.
But frankly lisp is such a non-multi-system language, it has a hard time to deal with external world by its nature. It can be done as lisp is really the god level programming language. But as said it is NOT used by the gods for a reason.
We need find a system level language to express ourselves so that we can stand in giants. We need giants but no need to be one.
TAOCP
Perfect book and if you don’t have a version - get it!
That and the art of computer programming
I picked up SICP expecting to read something really interesting or profound with the way it's been hyped up over the years however it's more of a how-to manual for working with Scheme/LISP and frankly that didn't interest me. Unfortunately most people have come to accept that LISP isn't a particularly effective way of programming even if some people get really excited by the idea of mutable and interchangeable data and code it's just not as powerful as they make it out to be and the obfuscation of program flow and execution and the lack of separation/delineation of data and code proves to be a hinderance more often than it is helpful. This doesn't discount LISP's contribution to computer science historically and how it's influenced modern day language design over the years, just that in my opinion LISP/SCHEME is more of a historical curiosity than a modern day guide to effective programming. (And certainly one that has no place as the introductory class at MIT). Anyway I've said something negative about SICP so prepare for this to be downvoted to the bottom :)
and CSAPP i think
> In fact, I’d go further and say that it’s the only computer science book of that age that I’d happily and usefully read again without it being just for historical interest: the content has barely aged at all. That’s not all that unusual for mathematics books, but it’s almost unheard of in computer science, where the ideas move so quickly and where much of what’s written about is ephemeral rather than foundational.
I recall that when MIT stopped teaching with SICP, one of the main claims was that programming now is often not about thinking abstractions through from first principles, and creating some isolated gem of composing definitions. Instead, we interact with and rely on a rich ecosystem of libraries and tools which often have individual quirks and discordant assumptions, and engineering then takes on a flavor of discovering and exploring the properties and limitations of those technologies.
I think now, (some) people also are at the point of not even directly learning about the limitations and capability of each tool in their toolbox, but leaning heavily on generative tools to suggest low-level tactics. I think this will lead to an even messier future, where library code which works on (possibly generated) unit tests will bear some fragile assumption which was never even realized in the head of the engineer that prompted for it, and will not only fail but will be incorporated in training data and generated in the future.
I recall that when MIT stopped teaching with SICP, one of the main claims was that programming now is often not about thinking abstractions through from first principles, and creating some isolated gem of composing definitions.
Which is a category mistake that they actually address in the lectures. SICP is not a programming course, it’s a computer science course. Computer science is not about computers, let alone programming, just as geometry is not about surveying instruments and astronomy is not about telescopes.
When they stopped teaching SICP — in response to the pressure to teach more modern tools — they abandoned their scientific principles to satisfy commercial concerns. They stopped teaching computer science and became a vocational school for the tech industry.
I'm fine with the claim that CS is not about computers as astronomy is not about telescopes, but there are pure CS courses that don't cover programming and do cover automata, turing machines, computability, complexity etc and don't cover programming. SICP is about programs in a running language rather than abstract idealized computations, and is centered around reading and writing programs as examples. I think its success stems from its grounding in exhibiting such examples that ordinarily would not become accessible to students so quickly.
If you watch the lecture videos they make it a point of emphasis that they’re not interested in teaching LISP and that the course is not a LISP course. To show running programs they had to select some programming language. This was a regrettable concession because some students will inevitably think the course is about that language and miss the general principles they’re trying to teach:
1. Primitive operations
2. Means of combination
3. Means of abstraction
These are the three main features of programming languages, according to Sussman and Abelson. They wanted us to stop bikeshedding over the superficial details and just look at a tool for these 3 features, then be able to implement the algorithms and data structures we already know. This is how you become a wizard who can cast magic spells.
> SICP is not a programming course, it’s a computer science course.
I don't see what you mean by this at all. Furthermore this doesn't strike me as a useful distinction when a) it doesn't cover most topics labeled by consensus as "computer science" and b) it very clearly does teach a great deal about programming.
Why not say it teaches computer science and programming skills? Why do these have to be exclusive? There's obviously a great deal of overlap in general.
I don't see what you mean by this at all
The goal of the course is not to teach programming skills, it's to teach computer science. The difference is explained quite thoroughly in the lectures. One might even say that answering the question "what is computer science?" is one of the core goals of the course and a major part of the philosophy of the professors who created the course.
The argument being made by the comparisons to geometry and astronomy is that in any discipline there is a difference between means and ends: what you are attempting to achieve is distinct from the tools you're using to achieve it. Furthermore, it's a mistake to believe that the discipline is all about the tools. No, the tools are the means, not the end.
> The goal of the course is not to teach programming skills, it's to teach computer science.
Who cares what the goal is? It teaches programming skills too. The intent is irrelevant and for the most part so too is the distinction (outside the american education system, anyway).
> Furthermore, it's a mistake to believe that the discipline is all about the tools.
Who outside the american education gives a damn about "the discipline", if that refers to anything meaningful outside the american education system in the first place? It's arbitrary and has no purpose or benefit aside from organizing the education system. This is a course that miraculously, against all odds, manages to teach useful skills in addition to jargon patterns of thought. Why not celebrate this?
Anyway, programming is a useful pedagogical tool for teaching CS. CS is a useful pedagogical tool for teaching programming. To brag about not teaching one is just hobbling your own insight into the value you provide students.
I myself have a CS degree from a prestigious institution and largely enjoyed my education. But this attitude you alude to is just jerking off for the sake of jerking off. Particularly in the case of SICP.
The quote was about "programming by poking" which I take as highly relevant to actual distributed software. It meant (1) systems are more built by integrating many components, and (2) for many reasons, the components are not understood by the integrator and (3) they must resort to experimentation to validate how things actually work.
Unless you have a TLA+ model of all your components and how they interact, I would argue you don't understand your distributed system either, for all inputs.
https://web.archive.org/web/20160505011527/http://www.poster...
You don't think they can address both needs? What are non-MIT schools teaching when they teach CS if it's not SICP? Is everyone else just a vocational school?
SICP isn’t the only way to teach CS, obviously, but I’ll be honest with you: some schools aren’t even trying. They just offer Java and Python programming courses and call it a day.
> They stopped teaching computer science and became a vocational school for the tech industry.
Sheldon always said that MIT is a trade school
I've witnessed how abandoning first principles undermines the evolution of a system. If our mental model of a system is not formalized into first principles (i.e. a high-level specification), then successive generations of engineers will have to re-learn those principles through trial-and-error. They'll introduce mutations and dependencies between the mutations-- and when they leave, the next generation of maintainers will repeat the process. Generations of mutations eventually create a brittle, calcified creature of a system which people fear to touch with a ten foot poll.
I imagine people who were taught SICP would be more respectful, if not inclined, towards a formal articulation of a system's principles.
This philosophy is described in depth in the original 1985 article https://gwern.net/doc/cs/algorithm/1985-naur.pdf and in more accessible language in https://www.baldurbjarnason.com/2022/theory-building/. You can also observe engineers opposing/misunderstanding the need for specification in https://news.ycombinator.com/item?id=42114874
I think just like traditional engineers have to learn physics, computer people should learn these fundamentals for exactly the reason you outline.
Then, when you hit the job market, you learn the ecosystem of what other engineers have built and you work in that context.
In this way, you can eventually reach extreme productivity. Just look at humanity's GDP over the last 200 years.
[dead]
its watever
I considered reading SICP recently but this changed my mind:
> It's old and feels old. originally in scheme, they recently re released the book in JavaScript which is more approachable to today's audiences and there are still good things in there about encapsulation and building dsls. ymmv. Though the language and programming design concepts hold up, we're playing at higher levels of abstraction on more powerful machines and consequently the examples sometimes seem too tiny and simple.
I had studied economics in a similar way, but learning slightly old/outdated ideas demotivated me - I was much more interested in learning what works and what's considered the best way to do things, not what had been considered a good idea at some point in the past.
I don't want to be a downer on SICP (especially since I haven't even read it), but I hope this info might help others (or elicit a strong refutation).
Scheme as basically an implementation of the untyped lambda calculus will eternally be a good frame work to think about the problems of computation in.
In the more practical area Racket (the most modern Scheme) has basically any practical functionality you would want, while amazingly remaining a platform for an incredible amount of experimentation in computation and programming language theory.
But SICP is a book that is for people interested in the study of computation what programming languages can be. If you're worried about getting a job in software it won't be all that useful, but it will remain a classic for anyone interested in engaging in creating the future of software.
Sure, SICP is not a good book for people wanting to do rote learning, imitation, 'best practice' while ignoring the history.
It's for people that would like to learn rather advanced programming techniques and foundational ideas in computer science.