Technology-independent software-development courses

I was recently asked something like the following question, which I am citing with a few minor edits.

We would like to do a course about SW development with Python, preferably an online course, so that we can start at our own pace.

We don’t want a Python course, but would instead like a course more about SW development. It would be great if it were in Python because we are comfortable with it.

The interesting topics would be:

• object-oriented programming
• functional programming
• design patterns
• good coding practices

As well as other important topics such as:

• Testing
• Documenting
• Version control
• Working in a team with version control

The course doesn’t have to contain all these topics. It can be also several courses or it can be toy-projects from somewhere.

Learning how to develop software

I have very little familiarity with courses as I’ve usually been tasked with figuring out how to do things before others have gotten to it. Of late, I’ve been teaching courses, not taking them.

So, how did I learn what I know about software development? When I started writing software, there was nothing available online, outside of a bunch of GeoCities pages (one of which was mine). MSDN was on CDs or local help files.

I read some books, OOSC and OOSC2, as well as the Gang of Four’s Design Patterns. I can’t remember what else, but that’s partly how I leveled up my skills. I had the great fortune of being able to build and work on large frameworks, from which I drew many lessons. I worked with very good people, who challenged me and taught me a lot.

Nowadays, I use DuckDuckGo as my online reference. I have developed a relatively advanced skill at searching for what I’m looking for. I very often get it within minutes. I almost never use videos.

a primary skill in software development is to be able to imagine what you should be looking for. That is, you don’t have to know how to do everything without looking it up, but you do have to imagine that it might exist.

For example, I don’t know how to write automated tests in Python, but I know that it should be possible. I know that I should figure that out very early in my experiments with Python. I know what to expect from an automated-testing environment. I know which settings to look for and expect.

That kind of knowledge transfers from one language or development environment to another. I know that I code-completion makes me faster, I know that I would like to avoid runtime errors—how can I best use Python to achieve those ends?

Online Courses

I took a quick look around for online courses, but was not immediately convinced that I am equipped to be able to distinguish between scams and actually worthwhile courses. Does the course even mention general software-development principles? How much time is allocated to that?

Udemy

The Complete Software Engineering Course with Python (Udemy) looks as follows:

Udemy Python Software Course

• Only about 1% of students even bothered to rate the course
• The pricing fees high pressure and scammy
• The course descriptions are barely in English

What about general programming?

Udemy Software Engineering Lecture: 9.5 minutes

Just over nine minutes? And you can’t even be bothered to describe it in something approaching well-written English? No, thanks.

PluralSight

The course Learning To Program − Part 2: Abstractions (PluralSight) looks a bit more professional, but it still has some quirks (especially for $29 per month).

PluralSight Details

• Overall more professional than Udemy, but also seems to have been incompletely translated from French
• Still, there’s definitely more time dedicated to core concepts

There is an assessment that you can take, but you have to sign up first.

PluralSight Assessment

Maybe PluralSight is able to tell you which courses you need, but I doubt it will err on the “you need fewer courses” side.

Dometrain

I’ve recently heard from a source I’ve been watching for a while that this course is quite good for C# developers: From Zero to Hero: Test-Driven Development in C# by Guilherme Ferreira. The person recommending it releases quite interesting/advanced videos on YouTube and has his own range of courses at DomeTrain.

How should software-development be taught?

How would I teach basic software-development principles? I would probably start with very abstract principles that try to answer the classic questions for “use cases”:

• Requirements: What are you trying to achieve?
• Actors: Who is trying to achieve it?
• Stakeholders: Are there other points of view than just the actor’s?

Which language?

A question people tend to start with is: which programming language should I use?

That’s the wrong question.

The applicability of programming languages to fields differ widely, but most languages have a large overlap in functionality. Where they differ is in the degree of runtime or library support for specific tasks.

For example, Python famously has a lot of libraries for number-crunching and data-analysis (although I feel that this advantage is grossly exaggerated) whereas it’s terrible for writing Windows GUI applications. C#/.NET has excellent web and desktop technology support. The Python runtime is notoriously slow (with essential libraries written in C++) whereas .NET is known as a very performant cross-platform runtime.

Do you see how quickly the conversation turns from “what can the language do?” to “what can the standard runtime/libraries/environment do?” That’s because you can do most tasks with most languages.

Instead, we want to think about this at a higher level. We want to,

• maximize useful expressiveness while minimizing harmful expressiveness.
• accommodate inherent complexity without introducing accidental complexity.
• express our intent explicitly in our programs.
• be able to discover and eliminate assumptions
• get compilation errors or warnings, not runtime errors.

Developer discipline

Programming languages exist on several spectra. One of these is “the degree of developer discipline required to use the language effectively and safely.”

What does that mean? For example, Python and JavaScript have a dynamic type system. While there are mechanisms, practices, and IDE support that you can use to set up guardrails missing in the language, but they are optional and Idiomatically written code in both of these languages tends not to use any of it. It’s the wild west, for the most part, with a lot of assumptions that nothing will ever go wrong.

More strict languages force you to consider all possibilities before your program even compiles or runs. For example, Haskell and Rust are famously picky. If you have a function that returns a value under certain conditions, those languages will make you explicitly indicate what to return when those conditions don’t hold. Forgiving languages will just use some default value, usually null or undefined.

This is called “happy path” programming because you only write the code for the hoped-for path through your use case. For example, the user selects a valid file with the expected data format with an acceptable length with no validation or processing errors, generating a data file to which the initiating user has access.

• Did the user click cancel? Not handled.
• Was the file missing? Not handled.
• Was the file in an unexpected encoding? Not handled.
• Did the file fail to validate? Not handled.
• Was the data empty? Not handled.
• Did processing crash? Not handled.
• Was the generated output not accessible to the user? Not handled.

Writing programs in this fashion is a dangerous thing to do with a strict language, and it’s even worse to do in a lax language.

Even the simplest software has many, many branches. The less your language or compiler or IDE reminds you of them, the more you have to fill that gap with developer discipline.

Important language/runtime/IDE features

To get more concrete, some good questions to consider are:

• Can you clearly describe and use types? (implicitly typed ≥ explicitly typed ≥ dynamically typed)
• Are types statically checked? (yes > no)
• Can you primarily work with early binding? E.g., how strongly idiomatic are virtual/late bindings?
• Are data and operations idiomatically merged or separate? Is there support for pure data structures (e.g. records vs. classes)
• What about discriminated unions? Range types? Can you avoid primitive obsession?
• Can you declare non-nullable references?
• Can you designate functions as pure?
• Is a functional programming style supported?
• Is there a way to make data or records immutable?
• To what degree can you optimize performance where needed?
• How does error-handling work?
• How concise can you be? What kind of abstraction mechanisms are there? Do you have to write a lot of boilerplate?
• How does I/O work? Is it a second-class citizen? (I.e., does the language enforce purity so hard that it make it a pain to read from a file? I’m looking at you, Haskell and Elm)
• What is the asynchronous programming model?
• How good are the error messages?

If these don’t make any sense to you, don’t worry. But they are questions that are important when you’re choosing a tool for building software.

Intent & handling events

The whole point of a programming language is to express intent. You indicate what you intend to happen when a given event occurs.

An programmer expresses an intent by writing that, “when this thing happens, I intend for this other thing to happen.”

For example,

• When a filename is provided on the command line, read the contents of the file in a given encoding, process it as lines of text, and save the results to another file.
• When the user clicks the screen in a particular place when the program is in a particular state (e.g., over a button displayed in a dialog box), then execute an operation.
• When an HTTP request arrives, then read the body, process it, and return an HTTP response with the results
• When a sensor triggers an interrupt, then change a GUI readout from red to green
• When the system sends a shutdown message, then close and flush all logs

Questions to consider

How do we choose a programming language? You’re not just choosing a programming language, you’re also implicitly deciding which subset of language features to use. This is predicated, of course, on knowing about these features. It’s best to inform yourself about what your language/libraries/runtime (let’s call it a software-development tool) can do for you—or find someone who is well-informed to help.

For each feature, you should ask yourself: how useful is it? Does it help me achieve my task?

Let’s take a look at high-level features of a software-development tool that may be important.

Maintainability

• By whom? What level of programmer?
• How much will a certain part need to change?

Comprehensibility

• By whom? What level of programmer?
• Are you applying the “rule of least power”? That is, are you using the simplest, most effective tool in your arsenal? To know this, you have to expand your arsenal … and then use it judiciously
• Low syntax noise

Discoverability

• Can you use the IDE and code-completion to learn the API?
• How much documentation do you need?
• How much on-boarding would a new developer need?
• How easy would it be to hand off to another person or team?

Learnability

• Even if not especially discoverable, how learnable is the code?
• Are the patterns consistent?
• Is the API clear?

Strictness / Correctness

Error-free is strongly encouraged or enforced

Complexity

Inherent vs. Accidental

Idiomatic

Is it written in a way that other programmers of this language or framework can easily understand?

Testability

• Can you write semantically useful tests?
• Is it easier to write a test than to debug?

Debuggability

• Can your debugger set a breakpoint on all of the important bits?
• Can you even debug it?
• Is there too much magic?
• Is there so much generalization that you can’t figure out what’s going on?

Observability

• Is there sufficient logging to figure out what’s going on without debugging? (This is important once the product is in the field.)
• What about error-handling? Are problems separated from errors?[1]
• Can the developer predict what’s happening and measure it?
• How’s your telemetry?
• Could the software be monitored if needed?

Performance

• Does it need to be fast?
• All of it? Or just parts?

For code designed to be reusable (libraries, frameworks), you can also consider:

Completeness

degree to which the definition/API captures all facets of the problem domain

Expressiveness

Concise and precise

Flexibility

Applicability to different problem domains

Which of the features above matters more depends on what you’re building. A one-off script doesn’t need to satisfy many of these features. A full-blown application that needs to be maintained for 10-20 years by different teams has to be much, much more careful.

Other articles

This isn’t the first time I’ve written about these ideas, so I’ve included links to other, similar articles below.

General Programming Practices

These articles discuss the topic of software-development on a similar level to the discussion above.

• Elegant Code vs.(?) Clean Code
• Programming-language Features: How much is too much?
• Applying the rule of least power in practice
• Framework Design: Programming Paradigms and Idioms

The articles below are more recent, are more-or-less on the same level, but are also more targeted.

• How to evaluate dependencies
• To crash or not to crash; that is the question
• Stop trying so hard to use pattern-matching
• The UI is an afterthought, a detail
• On the usefulness of containers like Docker
• The “Hustle” culture in Software Development

Encodo White papers archive

These white papers were written from 2006 to 2019 when I was still employed at Encodo Systems AG. They expand on recommended practices of specific facets of software development. They are presented in reverse-chronological order, but can be read in any order.

• DI, IOC and Containers (2019)
• Clean and Safe Code (2019)
• Testing as First-Class Citizen (2019)
• Code Reviews (2017)
• Document Everything (2017)
• Continuous Integration and Delivery (2017)
• Component-based Design (2006)
• Design by Contract (2006)
• Test-driven Development (2006)

Videos

This is a YouTube playlist I’ve maintained for years that I continuously update whenever I watch a video that I think would be interesting for other developers. It’s only technology videos, but it’s pretty eclectic (i.e., it’s language- and technology-agnostic).

Developer suggestions (YouTube)

Conclusion

Pace yourself. You can’t have everything all at once. Programming takes wisdom. Wisdom takes time. It takes practice. It comes, or it doesn’t. It takes different forms.

As Rainer Maria Rilke wrote in 1903[2],

“Forschen Sie jetzt nicht nach den Antworten, die Ihnen nicht gegeben werden können, weil Sie sie nicht leben könnten. Und es handelt sich darum, alles zu leben. Leben Sie jetzt die Fragen. Vielleicht leben Sie dann allmählich, ohne es zu merken, eines fernen Tages in die Antwort hinein.”

Good luck.