
What is Typed Python? Why is it important for Python developers today? How to can you get started?
Python is one of the most successful programming languages out there, with it recently overtaking Javascript as the most popular language on GitHub, according to the latest GitHub Octoverse report. The report emphasises the popularity of the language in the growing fields of AI, data science and scientific computing - fields where speedy experimentation and iteration are critical, and where developers are coming from a broad range of STEM backgrounds, not necessarily computer science. But as the Python community expands and projects grow from experiments to production systems, that same flexibility can become a liability.
That’s why today we’re going to talk about typed Python - what it is, why it’s become important for Python developers today, and how to get started using it to write higher quality, more reliable code.
Before we dive into why you should be using typed Python in your daily development lives, first we need to understand some core concepts and how we got here.
The classic Python programming language that you know and love is dynamically typed. What does that mean exactly? It means that types are determined at runtime, not when you write your code. Variables can hold any type of value, and you don't need to declare what type they are.
Here’s an example of dynamic typing in action:
x = 5 # x is an integer
x = "hello" # now x is a string
x = [1,2,3] # now x is a list
This behaviour is one of the things that sets Python apart from languages that are statically typed, like Java or C++, which require you to declare types from the get go:
int x = 5;
std::string x_str = "hello";
std::vector<int> x_vec = {1, 2, 3};
In the above example we can’t just reassign the variable x to a value of whatever type we want, it can only hold an integer because of the static typing nature of the C++ language.
The fact that Python is a dynamically typed language is one of the reasons it is so easy to use and popular amongst new and experienced programmers alike. It makes it easy to develop quick demos, experimental research and proof of concepts, without needing to spend precious development time declaring types. This flexibility has been instrumental in Python's adoption in AI, data science, and scientific computing, where researchers need to rapidly iterate and experiment with different approaches.
However… (surely you knew there was a “but” coming?)
We are quickly moving past the “proof-of-concept” phase for many of these industries. AI and machine learning efforts are actively being integrated into production applications, and with that comes production-level expectations of reliability and stability. Relying on dynamic typing opens these codebases up to a certain level of risk that may not be acceptable at the scale they are now expected to operate.
Cast your mind back to September 2014: Germany has just won the world cup, skinny jeans are still in fashion and Taylor Swift’s “Shake it Off” is number 1 on the charts. That same month PEP 484 was first created, proposing the addition of type hints to Python, and fundamentally changing how future developers would be able to write and maintain Python code.
With PEP 484’s acceptance and introduction in Python 3.5, developers could now use static type annotations to declare the expected data types of function arguments and return values, and subsequent PEPs have continually added more features to expand and refine Python's type system. Today you can write statically typed Python statements like this:
def my_func(x: int, y: str) -> bool:
z: str = str(x)
return z == y
The key innovation of PEP 484 was introducing a gradual type system that allows developers to slowly add type annotations over time without breaking existing code. The system works by:
Any type as an escape hatch that has all possible attributesAnyThis approach has meant developers can incrementally adopt typing, while still allowing them to take advantage of the default dynamic typing approach that makes Python so easy to work with and ideal for quick prototyping.
So why specifically should you start using type hints in your Python code? Python type hints offer a range of advantages that can significantly improve the quality, maintainability, and scalability of your codebase, at the same time making it easier for other developers to understand your code and collaborate with you.
Type hints assist static analysis tools in identifying mismatches and potential errors before the code is executed, allowing for early bug detection. Take the following example:
def add_numbers(a, b):
return a + b
...
add_numbers(3, "4") # Potential error
The above error might be easy to spot when you’re calling the function so close to where you’re defining it, but imagine you’re working across multiple files and/or with many lines of code separating them - suddenly it’s not so easy!
In comparison, if you’re using type hints in conjunction with a typechecking tool (such as Pyrefly or MyPy), you can catch this error much earlier - when you’re actually writing the code, rather than when it fails at runtime:
def add_numbers(a: int, b: int) -> int:
return a + b
...
add_numbers(3, "4") # a typechecker will catch this error at time of writing
Using a typechecker to highlight these types of errors also ensures you can catch an error like this even if you’ve missed this code path in your unit tests.
Another benefit of writing typed Python is that using function signatures and variable annotations provide clarity of intent for a given piece of code. In other words, it makes code easier to read and review. It makes refactoring safer and more predictable. It even helps new team members get up to speed quickly on what’s going on in your codebase without wasting their own time, or yours!
Take the following example, without type hints you have to carefully read the internal function code to understand what type of parameters will work and what will be returned:
def calculate_stats(data, weights):
total = 0
weighted_sum = 0
for i, value in enumerate(data):
if i < len(weights):
weighted_sum += value * weights[i]
total += weights[i]
avg = weighted_sum / total if total > 0 else 0
return avg, len(data)
With this version, you can tell instantly what type of arguments you should be passing and what you should expect to get back - saving precious dev time and just generally making your life easier:
def calculate_stats(data: list[float], weights: list[float]) -> tuple[float, int]:
total = 0
weighted_sum = 0
for i, value in enumerate(data):
if i < len(weights):
weighted_sum += value * weights[i]
total += weights[i]
avg = weighted_sum / total if total > 0 else 0
return avg, len(data)
I know I know - ideally all developers should be adding clear docstrings with every function they write, but we know in reality it doesn’t always shape up that way! Adding type hints is quicker than writing a typical docstring, won’t go stale (if enforced using a typechecker) and is better than no documentation at all. Modern Python typecheckers also have IDE extensions that include autocomplete functionality to make life easier.
One of the most important benefits of using type annotations in your code is that it helps you scale your code faster and with less risk. For developers today, the pipeline from experimental code to production systems moves faster than ever, especially in AI and machine learning workflows where research prototypes must quickly evolve into robust, scalable applications.
For example, say there is a team of data scientists that has just published their findings and now needs to operationalize their models. If their published code already includes type hints it makes it much easier, quicker and safer for an engineering team to step in and integrate that research into production applications. In situations like these, type annotations act as a contract between different stages of development, making it clear how data flows through complex, multi-step processing pipelines. This is particularly valuable in AI workflows where a single type mismatch, like passing a NumPy array where a PyTorch tensor is expected, can cause silent failures or performance degradation that only surfaces under production load.
So now you know what typed python is and why you should be doing it, how can you actually get started adding types to your code?
As a general rule of thumb, the earlier in a project you start adding type annotations the better. Type hints are much easier to add as you go than to retrofit across an entire codebase later.
As we’ve mentioned before, one of the great benefits of Python is that its dynamic typing default makes it very flexible and easy to get started with. So when you’re doing your initial experimentation and prototyping maybe you’re not thinking about making sure it’s type safe - and that’s ok! But as soon as you start to think your project might be going somewhere, if more than one person might be working on it, using it or just reading it, you should start adding type hints.
Choose and install a type checker that fits your needs. Typecheckers leverage the code annotations you write to provide important errors and warnings to ensure your codebase is type safe.
At Meta, we recommend Pyrefly, our new open-source type checker built in Rust. Pyrefly is designed to scale from small projects to massive codebases incredibly fast, while providing excellent developer experience. Read the Pyrefly documentation to understand configuration options and best practices, then start adding simple type hints to new functions before gradually working your way up to more complex scenarios.
You should also consider working with a typechecker that supports IDE integration to get real-time feedback as you write code. Pyrefly provides extensions for editors like VS Code, PyCharm, and Vim which will highlight errors and provide autocomplete suggestions based on your type annotations.
Adding your typechecker to your CI processes is also valuable for maintaining code quality at scale. You can configure your CI/CD pipeline to run type checking on every pull request, treating type errors as build failures.
Typing is one of those skills that gets better the more you practice it in your code, but there are also great resources out there for getting to grips with the functionality and diving deeper into the concepts:
So there you have it - a quick trip around the world of Python typing! By now you’ve hopefully learnt that type hints aren't just another Python feature to add to the long list of things you’ll definitely get round to implementing eventually - they're a practical investment in your code's future. The upfront effort of adding type hints pays dividends in reduced debugging sessions, smoother code reviews, and fewer production issues. Most importantly, they give you the confidence to refactor and scale your codebase without fear of breaking things in unexpected ways. Start small by adding type annotations to your next function, add a type checker to your workflow, and before you know it writing typed Python will be second nature. Your future self (and your users and teammates!) will thank you.
I actually don’t like python type hints!
At my work we have a jit compiler that requires type hints under some conditions.
Aside from that, I avoid them as much as possible. The reason is that they are not really a part of the language, they violate the spirit of the language, and in high-usage parts of code they quickly become a complete mess.
For example a common failure mode in my work’s codebase is that some function will take something that is indexable by ints. The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types! And you better believe that every single admissible type will eventually be fed to this function. Sometimes people will try to keep up, annotating it with a Union of the (growing) list of admissible types, but eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.
So, if the jit compiler needs the annotation I am happy to provide it, but otherwise I will proactively not provide any, and I will sometimes even delete existing annotations when they are devolving into silliness.
That's the same complaints people had about TypeScript in the beginning, when libraries such as Express used to accept a wide range of input options that would be a pain to express in types properly. If you look at where the ecosystem is now, though, you'll see proper type stubs, and most libraries get written in TS in the first place anyway. When editing TS code, you get auto-completion out of the box, even for deeply nested properties or conditional types. You can rely on types being what the compiler says they are, and runtime errors are a rarity now (in properly maintained code bases).
> The reason is that they are not really a part of the language, they violate the spirit of the language, and in high-usage parts of code they quickly become a complete mess.
I'll admit that this is what I hate Python, and it's probably this spirit of the language as you call it. I never really know what parameters a function takes. Library documentation often shows a few use cases, but doesn't really provide a reference; so I end up having to dig into the source code to figure it out on my own. Untyped and undocumented kwargs? Everywhere. I don't understand how someone could embrace so much flexibility that it becomes entirely undiscoverable for anyone but maintainers.
Because the flexibility has been a boon and not a problem. The problem only comes when you try to express everything in the type system, that is third party (the type checkers for it) and added on top.
It's a boon if the goal is to write code then go home. It's a loaded footgun if the goal is to compose a stack and run it in production within SLO.
Python type hints manage to largely preserve the flexibility while seriously increasing confidence in the correctness, and lack of crashing corner cases, of each component. There's really no good case against them at this point outside of one-off scripts. (And even there, I'd consider it good practice.)
As a side bonus, lack of familiarity with Python type hints is a clear no-hire signal, which saves a lot of time.
People say this all the time, but I've never seen any data proving it's true. Should be rather easy too, I'm at a big company and different teams use different languages. The strictly typed languages do to have fewer defects, and those teams don't ship features any faster than the teams using loosely typed languages.
What I've experienced is that other factors make the biggest difference. Teams that write good tests, have good testing environments, good code review processes, good automation, etc tend to have fewer defects and higher velocity. Choice of programming language makes little to no difference.
I think with types there is a risk of typing things too early or too strictly or types nudging one to go in a direction, that reduces the applicability and flexibility of the final outcome. Some things can be difficult to express in types and then people choose easier to type solutions, that are not as flexible and introduce more work later, when things need to change, due to that inflexibility or limited applicability.
>It's a boon if the goal is to write code then go home. It's a loaded footgun if the goal is to compose a stack and run it in production within SLO.
Never has been an issue in practice...
Did you forget /s at the end of this?
I work at big tech and the number of bad deploys and reverts I've seen go out due to getting types wrong is in the hundreds. Increased type safety would catch 99% of the reverts I've seen.
Also have fun depending on libraries 10 years old as no one likes upgrades over fear of renames.
Ops type here, I’ve got multiple stories where devs have screwed up with typing and it’s caused downstream problems.
> Because the flexibility has been a boon and not a problem
Well, you could say that the problem in this case was the lack of documentation, if you wanted. The type signature could be part of the documentation, from this point of view.
Let me give a kind-of-concrete example: one year I was working through a fast.ai course. They have a Python layer above the raw ML stuff. At the time, the library documentation was mediocre: the code worked, there were examples, and the course explained what was covered in the course. There were no type hints. It's free (gratis), I'm not complaining. However, once I tried making my own things, I constantly ran into questions about "can this function do X" and it was really hard to figure out whether my earlier code was wrong or whether the function was never intended to work with the X situation. In my case, type hints would have cleared up most of the problems.
> the lack of documentation
If the code base expects flexibility, trusting documentation is the last thing you'd want to do. I know some people live and die by the documentation, but that's just a bad idea when duck typing or composition is heavily used for instance, and documentation should be very minimal in the first place.
When a function takes a myriad of potential input, "can this function do X" is an answer you get by reading the function or the tests, not the prose on how it was intended 10 years ago or how some other random dev thinks it works.
Documentation doesn’t have to be an essay. A simple, automatically generated reference with proper types goes a long way to tell me „it can do that“ as opposed to „maybe it works lol“. That’s not the level of engineering quality I’m going for in my work.
This whole discussion is about how you might not want to be listing every single types a function accepts. I also kinda wonder how you automatically generate that for duck typing.
Generally using the Protocol[1] feature
from typing import Protocol
class SupportsQuack(Protocol):
def quack(self) -> None: ...
This of course works with dunder methods and such. Also you can annotate with @runtime_checkable (also from typing) to make `isinstance`, etc work with itYou're then creating a Protocol for every single function that could rely on some duck typing.
Imagine one of your function just wants to move an iterator forward, and another just wants the current position. You're stuck with either requiring a full iterator interface when only part of it is needed or create one protocol for each function.
In day to day life that's dev time that doesn't come back as people are now spending time reading the protocol spaghetti instead of reading the function code.
I don't deny the usefulness of typing and interfaces in stuff like libraries and heavily used common components. But that's not most of your code in general.
For the collections case in particular, you can use the ABCs for collections that exist already[1]. There's probably in your use case that satisfies those. There's also similar things for the numeric tower[2]. SupportsGE/SupportsGT/etc should probably be in the stdlib but you can import them from typeshed like so
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from _typeshed import SupportsGT
---In the abstract sense though, most code in general can't work with anything that quack()s or it would be incorrect to. The flip method on an penguin's flipper in a hypothetical animallib would probably have different implications than the flip method in a hypothetical lightswitchlib.
Or less by analogy, adding two numbers is semantically different than adding two tuples/str/bytes or what have you. It makes sense to consider the domain modeling of the inputs rather than just the absolute minimum viable to make it past the runtime method checks.
But failing that, there's always just Any if you legitimately want to allow any input (but this is costly as it effectively disables type checking for that variable) and is potentially an indication of some other issue.
[1]: https://docs.python.org/3.14/library/collections.abc.html
> You're then creating a Protocol for every single function that could rely on some duck typing.
No, you are creating a Protocol (the kind of Python type) for every protocol (the descriptive thing the type represents) that is relied on for which an appropriate Protocol doesn’t already exist. Most protocols are used in more than one place, and many common ones are predefined in the typing module in the standard library.
Except Typescript embraces duck typing. You can say "accept any object with a quack() method", for example, and it'll accept an unexpected quacking parrot. It can even tell when two type definitions are close enough and merge them.
So does Python. They're called protocols. [0]
> Except Typescript embraces duck typing.
So does Python:
It's not duck typing if you have to declare the type...
Kind of, depends on the compiler configuration.
Doesn't Go also use structural typing?
I like Python a lot, and have been using it for personal projects since about 2010. It was only once I started working and encountering long-lived unfamiliar Python codebases regularly that I understood the benefits of type hints. It's not fun to have to trace through 5 or 6 different functions to try to figure out what type is being passed in or returned from something. It's even less fun to find out that someone made a mistake and it's actually two different incompatible things depending on the execution path.
That era of Python codebases were miserable to work in, and often ended up in the poorly though out "we don't know how this works and it has too many bugs, let's just rewrite it" category.
> It's not fun to have to trace through 5 or 6 different functions to try to figure out what type is being passed in or returned from something.
My position is that what is intended must be made clear between type hints and the docstring. Skipping this makes for difficult to read code and has no place in a professional setting in any non-trivial codebase.
This doesn't require type hints to achieve. :param and :rtype in the docstring are fine if type hints aren't present, or for complex cases, plain English in the docstring is usually better.
:param and :rtype are type hints, just type hints that cannot be validated by tooling and are guaranteed to go out of sync with the code eventually.
Proper type hints are typically very easy to add if the codebase is not a mess that passes things around far and wide with no validation. If it is, the problem is not with the type hints.
I agree, although I've found that correct and comprehensive use of the doctoring for this purpose has not existed in the environments I've worked in, or the open source codebases I have needed to understand. Something about type hinting makes people more likely to do it.
I am sorry, but whats wrong with doing something like, `print(type(var)); exit()` and just running it once instead of digging through 5-6 stack frames?
Sometimes a function's input or return type can vary depending on the execution path? Also, inserting print statements is often not practical when working on web backend software which is kind of a big thing nowadays. If you can run the service locally, which is not a given, dependencies get mocked out and there's no guarantee that your code path will execute or that the data flowing through it will be representative.
They don’t violate the spirit of the language. They are optional. They don’t change the behaviour at runtime.
Type annotations can seem pointless indeed if you are unwilling to learn how to use them properly. Using a giant union to type your (generic) function is indeed silly, you just have to make that function generic as explained in another comment or I guess remove the type hints
> They don’t violate the spirit of the language. They are optional.
That in itself violates the spirit of the language, IMO. “There should be one obvious way to do it”.
Well, precisely:
- There is one obvious way to provide type hints for your code, it’s to use the typing module provided by the language which also provides syntax support for it.
- You don’t have to use it because not all code has to be typed
- You can use formatted strings, but you don’t have to
- You can use comprehensions but you don’t have to
- You can use async io, but you don’t have to. But it’s the one obvious way to do it in python
The obvious way to annotate a generic function isn’t with a giant Union, it’s with duck typing using a Protocol + TypeVar. Once you known that, the obvious way is… pretty obvious.
The obvious way not be bothered with type hints because you don’t like them is not to use them!
Python is full of optional stuff, dataclasses, named tuples, meta programming, multiple ancestor inheritance. You dont have to use these features, but there are only one way to use them
> but there are only one way to use them
Optional nature of those features conflicts with this statement. As optionality means two ways already.
classes are optional in python, does that violate the spirit?
"There should only be one way to do it" has not really been a thing in Python for at least the last decade or longer. It was originally meant as a counterpoint to Perl's "there's more than one way to do it," to show that the Python developers put a priority on quality and depth of features rather than quantity.
But times change and these days, Python is a much larger language with a bigger community, and there is a lot more cross-pollination between languages as basic philosophical differences between the most popular languages steadily erode until they all do pretty much the same things, just with different syntax.
> "There should only be one way to do it" has not really been a thing in Python for at least the last decade or longer.
It never was a thing in Python, it is a misquote of the Zen of Python that apparently became popular as a reaction against the TMTOWTDI motto of the Perl community.
Not misquoted, paraphrased. I didn't feel like bothering to check the output of "import this" before posting.
the whole language violates this principle tbh, so it's very in spirit
Yeah that ship sailed some time before they added a third way to do templated string interpolation.
How so? There is one way to do it. If you want typing, you use type hints. You wouldn't say that, say, functions are unpythonic because you can either use functions or not use them, therefore there's two ways to do things, would you?
And Python failed at that decades ago. People push terribly complicated, unreadable code under the guise of Pythonic. I disagree with using Pythonic as reasoning for anything.
This is a popular misquote from the Zen of Python. The actual quote is “There should be one—and preferably only one—obvious way to do it.”
The misquote shifts the emphasis to uniqueness rather than having an obvious way to accomplish goals, and is probably a result of people disliking the “There is more than one way to do it” adage of Perl (and embraced by the Ruby community) looking to the Zen to find a banner for their opposing camp.
And Python failed at that decades ago. People push terribly complicated, unreadable code under the guise of Pythonic. I disagree with using Pythonic as reasoning for anything.
on that note, which is better, using `map()` or a generator expression?
Actually in Python it can. Since the type hints are accessible at runtime, library authors can for example change which values in kwargs are allowed based on the type of the argument.
So on the language level it doesn’t directly change the behavior, but it is possible to use the types to affect the way code works, which is unintuitive. I think it was a bad decision to allow this, and Python should have opted for a TypeScript style approach.
You can make it change the behaviour at runtime is different than it changes the behaviour at runtime I think?
Lots of very useful tooling such as dataclasses and framework like FastAPI rely on this and you're opinion is that it's a bad thing why?
In typescript the absence of type annotations reflection at runtime make it harder to implement things that people obviously want, example, interop between typescript and zod schemas. Zod resorts instead to have to hook in ts compiler to do these things.
I'm honestly not convinced Typescript is better in that particular area. What python has opted for is to add first class support for type annotations in the language (which Javascript might end up doing as well, there are proposals for this, but without the metadata at runtime). Having this metadata at runtime makes it possible to implement things like validation at runtime rather than having to write your types in two systems with or without codegen (if Python would have to resort to codegen to do this, like its necessary in typescript, I would personally find this less pythonic).
I think on the contrary it allows for building intuitive abstractions where typescript makes them harder to build?
Yeah, but then you get into the issues with when and where generic types are bound and narrowed, which can then make it more complicated, at which point one might be better off stepping back, redesigning, or letting go of perfect type hint coverage, for dynamic constructs, that one couldn't even write in another type safe language.
I don’t know anything about your jit compiler, but generally the value I get from type annotations has nothing to do with what they do at runtime. People get so confused about Python’s type annotations because they resemble type declarations in languages like C++ or Java. For the latter, types tell the compiler how to look up fields on, and methods that apply to, an object. Python is fine without that.
Python’s types are machine-checkable constraints on the behavior of your code.. Failing the type checker isn’t fatal, it just means you couldn’t express what you were doing in terms it could understand. Although this might mean you need to reconsider your decisions, it could just as well mean you’re doing something perfectly legitimate and the type checker doesn’t understand it. Poke a hole in the type checker using Any and go on with your day. To your example, there are several ways described in comments by me and others to write a succinct annotation, and this will catch cases where somebody tries to use a dict keyed with strings or something.
Anyway, you don’t have to burn a lot of mental energy on them, they cost next to nothing at runtime, they help document your function signatures, and they help flag inconsistent assumptions in your codebase even if they’re not airtight. What’s not to like?
So the type is anything that implements the index function ([], or __getitem__), I thnink that's a Sequence, similar to Iterable.
>from typing import Sequence
>def third(something: Sequence):
> return indexable[3]
however if all you are doing is just iterate over the thing, what you actually need is an Iterable
>from typing import Iterable
>def average(something:Iterable):
> for thing in something:
> ...
Statistically, the odds of a language being wrong, are much lower than the programmer being wrong. Not to say that there aren't valid critiques of python, but we must think of the creators of programming languages and their creations as the top of the field. If a 1400 chess elo player criticizes Magnus Carlsen's chess theory, it's more likely that the player is missing some theory rather than he found a hole in Carlsen's game, the player is better served by approaching a problem with the mentality that he is the problem, rather than the master.
> we must think of the creators of programming languages and their creations as the top of the field
The people at the top of the type-system-design field aren’t working on Python.
> So the type is anything that implements the index function ([], or __getitem__), I thnink that's a Sequence
Sequence involves more than just __getitem__ with an int index, so if it really is anything int indexable, a lighter protocol with just that method will be more accurate, both ar conveying intent and at avoiding needing to evolve into an odd union type because you have something that a satisfies the function’s needs but not the originally-defined type.
That is sort of ironic because the Pythonistas did not leave out any opportunity to criticize Java. Java was developed by world class experts like Gosling and attracted other type experts like Philip Wadler.
No world class expert is going to contribute to Python after 2020 anyway, since the slanderous and libelous behavior of the Steering Council and the selective curation of allowed information on PSF infrastructure makes the professional and reputational risk too high. Apart from the fact that Python is not an interesting language for language experts.
Google and Microsoft have already shut down several failed projects.
>"Guido: Java is a decent language," 1999
I get the idea that Python and Java went in opposite directions. But I'm not aware of any fight between both languages. I don't think that's a thing either.
Regarding stuff that happens in the 2020. Python was developed in the 90s, python 3 was launched in 2008. Besides some notable PEPs like type hints, WSGI, the rest of development are footnotes. The same goes for most languages (with perhaps the exception of the evergrowing C++), languages make strong bc guarantees and so the bulk of their innovation comes from the early years.
Whatever occurs in the 20th and 30th year of development is unlikely to be revolutionary or very significant. Especially ignoreable is the drama that might emerge in these discussions, slander, libel inter-language criticism?
Just mute that out. I've read some news about some communities like Ruby on Rails or Nix that become overtaken by people and discussions of political nature rather than development, they can just be ignored I think.
> Google and Microsoft have already shut down several failed projects
Could you elaborate on this?
Sure: Google fired the Python language team in 2024 that contained a couple of the worst politicians who were later involved in slandering Tim Peters.
Before that, Google moved heavily from Python to Go.
Microsoft fired the "Faster CPython Team" this year.
It’s unlikely those layoffs are related to that, but rather the industry at large and end of zirp. Those type of folks are common in bigtech companies as well.
For example the dart/flutter team was decimated as well.
>you better believe that every single admissible type will eventually be fed to this function
That's your problem right there. Why are random callers sending whatever different input types to that function?
That said, there are a few existing ways to define that property as a type, why not a protocol type "Indexable"?
>why not a protocol type
it was a sin that python's type system was initially released as a nominal type system. they should have been the target from day one.
being unable to just say "this takes anything that you can call .hello() and .world() on" was ridiculous, as that was part of the ethos of the dynamically typed python ecosystem. typechecking was generally frowned upon, with the idea that you should accept anything that fit the shape the receiving code required. it allowed you to trivially create resource wrappers and change behaviors by providing alternate objects to existing mechanisms. if you wanted to provide a fake file that read from memory instead of an actual file, it was simple and correct.
the lack of protocols made hell of these patterns for years.
I disagree. I think, if the decision was made today, it probably would have ended up being structural, but the fact that it isn't enables (but doesn't necessarily force) Python to be more correct than if it weren't (whereas forced structural typing has a certain ceiling of correctness).
Really it enabled the Python type system to work as well as it does, as opposed to TypeScript, where soundness is completely thrown out except for some things such as enums
Nominal typing enables you to write `def ft_to_m(x: Feet) -> Meters: and be relatively confident that you're going to get Feet as input and Meters as output (and if not, the caller who ignored your type annotations is okay with the broken pieces).
The use for protocols in Python in general I've found in practice to be limited (the biggest usefulness of them come from the iterable types), when dealing with code that's in a transitional period, or for better type annotations on callables (for example kwargs, etc).
>The use for protocols in Python in general I've found in practice to be limited (the biggest usefulness of them come from the iterable types)
Most Python's dunder methods make it so you can make "behave alike" objects for all kinds of behaviors, not just iterables
TypeScript sacrificed soundness to make it easier to gradually type old JS code and to allow specific common patterns. There is no ceiling for correctness of structural typing bar naming conflicts.
AFAIK, Python is missing a fully-featured up to date centralized documentation on how to use type annotations.
The current docs are "Microsoft-like", they have everything, spread through different pages, in different hierarchies, some of them wrong, and with nothing telling you what else exists.
> That's your problem right there. Why are random callers sending whatever different input types to that function?
Because it’s nice to reuse code. I’m not sure why anyone would think this is a design issue, especially in a language like Python where structural subtyping (duck typing) is the norm. If I wanted inheritance soup, I’d write Java.
Ironically, that’s support for structural subtyping is why Protocols exist. It’s too bad they aren’t better and the primary way to type Python code. It’s also too bad that TypedDict actively fought duck typing for years.
Why can’t you re-use it with limited types? If the types are too numerous/hard to maintain it seems like the same would apply to the runtime code.
Because it’s nice to reuse code. It’s virtually never the case that a function being compatible with too many types is an issue. The issue is sometimes that it isn’t clear what types will be compatible with a function, and people make mistakes.
Python’s type system is overall pretty weak, but with any static language at least one of the issues is that the type system can’t express all useful and safe constructs. This leads to poor code reuse and lots of boilerplate.
>It’s virtually never the case that a function being compatible with too many types is an issue
This kind of accidental compatibility is a source of many hard bugs. Things appear to work perfectly, then at some point it does something subtly different, until it blows up a month later
if the piece of code in question is so type independent, then either it should be generic or it's doing too much
Yes. It's not the type system that's broken, it's the design. Fix the design, and the type system works for you, not against you.
> Why are random callers sending whatever different input types to that function?
Probably because the actual type it takes is well-understood (and maybe even documented in informal terms) by the people making and using it, but they just don’t understand how to express it in the Python type system.
> For example a common failure mode in my work’s codebase is that some function will take something that is indexable by ints. The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types! And you better believe that every single admissible type will eventually be fed to this function. Sometimes people will try to keep up, annotating it with a Union of the (growing) list of admissible types, but eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.
You are looking for protocols. A bit futzy to write once but for a heavily trafficked function it's woth it.
If your JIT compiler doesn't work well with protocols... sounds like a JIT problem not a Python typing problem
In my experience, the right tooling makes Python typing a big win. Modern IDEs give comprehensive real-time feedback on type errors, which is a big productivity boost and helps catch subtle bugs early (still nowhere near Rust, but valuable nonetheless). Push it too far though, and you end up with monsters like Callable[[Callable[P, Awaitable[T]]], TaskFunction[P, T]]. The art is knowing when to sprinkle types just enough to add clarity without clutter.
When you hit types like that type aliases come to the rescue; a type alias combined with a good docstring where the alias is used goes a long way
On the far end of this debate you end up with types like _RelationshipJoinConditionArgument which I'd argue is almost more useless than no typing at all. Some people claim it makes their IDE work better, but I don't use an IDE and I don't like the idea of doing extra work to make the tool happy. The opposite should be true.
sqlalchemy.orm.relationship(argument: _RelationshipArgumentType[Any] | None = None, secondary: _RelationshipSecondaryArgument | None = None, *, uselist: bool | None = None, collection_class: Type[Collection[Any]] | Callable[[], Collection[Any]] | None = None, primaryjoin: _RelationshipJoinConditionArgument | None = None, secondaryjoin: _RelationshipJoinConditionArgument | None = None, back_populates: str | None = None, order_by: _ORMOrderByArgument = False, backref: ORMBackrefArgument | None = None, overlaps: str | None = None, post_update: bool = False, cascade: str = 'save-update, merge', viewonly: bool = False, init: _NoArg | bool = _NoArg.NO_ARG, repr: _NoArg | bool = _NoArg.NO_ARG, default: _NoArg | _T = _NoArg.NO_ARG, default_factory: _NoArg | Callable[[], _T] = _NoArg.NO_ARG, compare: _NoArg | bool = _NoArg.NO_ARG, kw_only: _NoArg | bool = _NoArg.NO_ARG, lazy: _LazyLoadArgumentType = 'select', passive_deletes: Literal['all'] | bool = False, passive_updates: bool = True, active_history: bool = False, enable_typechecks: bool = True, foreign_keys: _ORMColCollectionArgument | None = None, remote_side: _ORMColCollectionArgument | None = None, join_depth: int | None = None, comparator_factory: Type[RelationshipProperty.Comparator[Any]] | None = None, single_parent: bool = False, innerjoin: bool = False, distinct_target_key: bool | None = None, load_on_pending: bool = False, query_class: Type[Query[Any]] | None = None, info: _InfoType | None = None, omit_join: Literal[None, False] = None, sync_backref: bool | None = None, **kw: Any) → Relationship[Any]You can use a Protocol type for that, makes a lot mote sense than nominal typing for typing use case.
Exactly, sounds like misuse of unions.
Although Python type hints are not expressive enough.
No idea about Python type system, but doesn't it have anything like this?
interface IntIndexable {
[key: number]: any
}It does!
You can specify a protocol like this:
class IntIndexable(Protocol[T]):
def __getitem__(self, index: int, /) -> T: ...
(Edit: formatting)The syntax is definitely harder to grasp but if the mechanism is there, I guess the parent poster's concern can be solved like that.
Although I understant that it might have been just a simplified example. Usually the "Real World" can get very complex.
> The syntax is definitely harder to grasp
Yes it is. I believe the reason is that this is all valid python while typescript is not valid javascript. Also, python's type annotations are available at runtime (eg. for introspection) while typescript types aren't.
That said, typescript static type system is clearly both more ergonomic and more powerful than Python's.
> eventually the list will become silly and the function will earn a # pyre-ignore annotation. This defeats the whole point of the pointless exercise.
No, this is the great thing about gradual typing! You can use it to catch errors and provide IDE assistance in the 90% of cases where things have well-defined types, and then turn it off in the remaining 10% where it gets in the way.
I feel pretty similarly on this. Python’s bolted on type system is very poor at encoding safe invariants common in the language. It’s a straight jacketed, Java-style OOP type system that’s a poor fit for many common Python patterns.
I would love it if it were better designed. It’s a real downer that you can’t check lots of Pythonic, concise code using it.
It sounds like that function is rightfully eligible to be ignored or to use the Any designation. To me that's why the system is handy. For functions that have specific inputs and outputs, it helps developers keep things straight and document code.
For broad things, write Any or skip it.
from typing import Protocol, TypeVar
T_co = TypeVar("T_co", covariant=True)
class Indexable(Protocol[T_co]): def __getitem__(self, i: int) -> T_co: ...
def f(x: Indexable[str]) -> None: print(x[0])
I am failing to format it proprely here, but you get the idea.
Just fyi: https://news.ycombinator.com/formatdoc
> Text after a blank line that is indented by two or more spaces is reproduced verbatim. (This is intended for code.)
If you'd want monospace you should indent the snippet with two or more spaces:
from typing import Protocol, TypeVar
T_co = TypeVar("T_co", covariant=True)
class Indexable(Protocol[T_co]):
def __getitem__(self, i: int) -> T_co: ...
def f(x: Indexable[str]) -> None:
print(x[0])I give Rust a lot of points for putting control over covariance into the language without making anyone remember which one is covariance and which one is contravariance.
One of the things that makes typing an existing codebase difficult in Python is dealing with variance issues. It turns out people get these wrong all over the place in Python and their code ends up working by accident.
Generally it’s not worth trying to fix this stuff. The type signature is hell to write and ends up being super complex if you get it to work at all. Write a cast or Any, document why it’s probably ok in a comment, and move on with your life. Pick your battles.
Kotlin uses "in" and "out": https://kotlinlang.org/docs/generics.html
Co- means with. Contra- means against. There are lots of words with these prefixes you could use to remember (cooperate, contradict, etc.).
There is also bunch of prepackaged types, such as collections.abc.Sequence that could be used in this case.
Sequence does not cut it, since the op mentioned int indexed dictionaries. But yeah.
Sequence[SupportsFloat] | Mapping[int,SupportsFloat]
Whether or not you explicitly write out the type, I find that functions with this sort of signature often end up with code that checks the type of the arguments at runtime anyway. This is expensive and kind of pointless. Beware of bogus polymorphism. You might as well write two functions a lot of the time. In fact, the type system may be gently prodding you to ask yourself just what you think you’re up to here.> Sequence[SupportsFloat] | Mapping[int,SupportsFloat]
This is really just the same mistake as the original expanding union, but with overly narrow abstract types instead of overly narrow concrete types. If it relies on “we can use indexing with an int and get out something whose type we don’t care about”, then its a Protocol with the following method:
def __getitem__(self, i: int, /) -> Any: ...
More generally, even if there is a specific output type when indexing, or the output type of indexing can vary but in a way that impacts the output or other input types of the function, it is a protocol with a type parameter T and this method: def __getitem__(self, i: int, /) -> T: ...
It doesn’t need to be union of all possible concrete and/or abstract types that happen to satisfy that protocol, because it can be expressed succinctly and accurately in a single Protocol.As of Python 3.12, you don’t need separately declared TypeVars with explicit variance specifications, you can use the improved generic type parameter syntax and variance inference.
So, just:
class Indexable[T](Protocol):
def __getitem__(self, i: int,/) -> T: ...
is enough.> something that is indexable by ints. > ...Dict[int, Any]...
If that is exactly what you want, then define a Protocol: from __future__ import annotations from typing import Protocol, TypeVar
T = TypeVar("T")
K = TypeVar("K")
class GetItem(Protocol[K, T]):
def __getitem__(self, key: K, /) -> T: ...
def first(xs: GetItem[int, T]) -> T:
return xs[0]
Then you can call "first" with a list or a tuple or a numpy array, but it will fail if you give it a dict. There is also collections.abc.Sequence, which is a type that has .__getitem__(int), .__getitem__(slice), .__len__ and is iterable. There are a couple of other useful ones in collections.abc as well, including Mapping (which you can use to do Mapping[int, t], which may be of interest to you), Reversible, Callable, Sized, and Iterable.Sounds like the ecosystem needs an "indexable" type annotation. Make it an "indexable<int>" for good measure.
Right, this was my thought.
Can’t you just use a typing.Protocol on __getitem__ here?
https://typing.python.org/en/latest/spec/protocol.html
Something like
from typing import Protocol
class Indexable(Protocol):
def __getitem__(self, i: int) -> Self: ...
Though maybe numpy slicing needs a bit more work to supportIndeed.
IMO, the trick to really enjoying python typing is to understand it on its own terms and really get comfortable with generics and protocols.
That being said, especially for library developers, the not-yet-existant intersection type [1] can prove particularly frustrating. For example, a very frequent pattern for me is writing a decorator that adds an attribute to a function or class, and then returns the original function or class. This is impossible to type hint correctly, and as a result, anywhere I need to access the attribute I end up writing a separate "intersectable" class and writing either a typeguard or calling cast to temporarily transform the decorated object to the intersectable type.
Also, the second you start to try and implement a library that uses runtime types, you've come to the part of the map where someone should have written HERE BE DRAGONS in big scary letters. So there's that too.
So it's not without its rough edges, and protocols and overloads can be a bit verbose, but by and large once you really learn it and get used to it, I personally find that even just the value of the annotations as documentation is useful enough to justify the added work adding them.
Slicing is totally hintable as well.
Change the declaration to:
def __getitem__(self, i: int | slice)
Though to be honest I am more concerned about that function that accepts a wild variety of objects that seem to be from different domains...
I'd guess inside the function is a HUGE ladder of 'if isinstance()' to handle the various types and special processing needed. Which is totally reeking of code smell.
>The type could be anything, it could be List, Tuple, Dict[int, Any], torch.Size, torch.Tensor, nn.Sequential, np.ndarray, or a huge host of custom types!
That's not how you are supposed to use static typing? Python has "protocols" that allows for structural type checking which is intended for this exact problem.
You explained some hyper niche instance where type hints should be ignored. 99% of the time, they are extremely helpful.
It's not even a niche instance, protocols solve their problem lol.
Can't you define your own hint for "type that has __getitem__ taking int"?
The way I understand parent is that such a type would be too broad.
The bigger problem is that the type system expressed through hints in Python is not the type system Python is actually using. It's not even an approximation. You can express in the hint type system things that are nonsense in Python and write Python that is nonsense in the type system implied by hints.
The type system introduced through typing package and the hints is a tribute to the stupid fashion. But, also, there is no syntax and no formal definitions to describe Python's actual type system. Nor do I think it's a very good system, not to the point that it would be useful to formalize and study.
In Russian, there's an expression "like a saddle on a cow", I'm not sure what the equivalent in English would be. This describes a situation where someone is desperately trying to add a desirable feature to an exiting product that ultimately is not compatible with such a feature. This, in my mind, is the best description of the relationship between Python's actual type system and the one from typing package.
> In Russian, there's an expression "like a saddle on a cow", I'm not sure what the equivalent in English would be
“To fit a square peg into a round hole”
Close but not the same. In Russian, the expression implies an "upgrade", a failed attempt at improving something that either doesn't require improvement or cannot be improved in this particular way. This would be a typical example of how it's used: "I'm going to be a welder, I need this bachelor's degree like a saddle on a cow!".
"Lipstick on a pig"? Although that's quite more combative than the Russian phrase.
Yeah... this seems like it would fit the bill nicely. At least, this is the way I'd translate it if I had to. Just didn't think about it.
the issue of having multiple inputs able to be indexable by ints, is exactly why i prefer that type hints remain exactly as "hints" and not as mandated checks. my philosophy for type hints is that they are meant to make codebases easier to understand without getting into a debugger. their functional equivalence should be that of comments. it's a cleaner more concise way of describing a variable instead of using a full on docstring.
though maybe there's a path forward to give a variable a sort of "de-hint" in that in can be everything BUT this type(i.e. an argument can be any indexable type, except a string)
>though maybe there's a path forward to give a variable a sort of "de-hint" in that in can be everything BUT this type
I think this is called a negation type, and it acts like a logical NOT operator. I'd like it too, and I hear that it works well with union types (logical OR) and intersection types (logical AND) for specifying types precisely in a readable way.
Define a protocol[0] that declares it implements `__getitem__` and type annotate with that protocol. Whatever properties are needed inside the function can be described in other protocols.
These are similar to interfaces in C# or traits in Rust - you describe what the parameter _does_ instead of what it _is_.
I like your point! I think the advantage in its light is this: People often use Python because it's convention in the domain, the project already uses it, or it's the language the rest of the team uses. So, you are perhaps violating the spirit, but that's OK. You are making the most of tools available. It's not the Platonic (Pythonic??) ideal, but good enough.
Isn't this supported by typing.SupportsIndex? https://docs.python.org/3/library/typing.html#typing.Support...
Mind you, I haven't used it before, but it feels very similar to the abstract Mapping types.
Oops, thanks for the correction. That's on me for drive-by commenting.
Why not do Indexable = Any and pass that? Even if it doesn't help your jit or the IDE, at least it is more explicit than
def lol(blarg): # types? haha you wish. rtfc you poor sod. Pytharn spirit ftw!!!
...
return omg[0].wtf["lol freedom"].pwned(Good.LUCK).figuring * out> The reason is that they are not really a part of the language, they violate the spirit of the language
This is a good way of expressing my own frustration with bolting strong typing on languages that were never designed to have it. I hate that TypeScript has won out over JavaScript because of this - it’s ugly, clumsy, and boilerplatey - and I’d be even more disappointed to see the same thing happen to the likes of Python and Ruby.
My background is in strongly typed languages - first C++, then Java, and C# - so I don’t hate them or anything, but nowadays I’ve come to prefer languages that are more sparing and expressive with their syntax.
I mean, you can just... Not annotate something if creating the relevant type is a pain. Static analysis \= type hints, and even then...
Besides, there must be some behavior you expect from this object. You could make a type that reflects this: IntIndexable or something, with an int index method and whatever else you need.
This feels like an extremely weak argument. Just think of it as self-enforcing documentation that also benefits auto-complete; what's not to love? Having an IntIndexable type seems like a great idea in your use case.
And you better believe that every single admissible type
This is exactly why I hate using Python.For this very specific example, isn't there something like "Indexable[int]"?
why not a protocol with getitem with an int arg?
This is like saying you don’t like nails because you don’t understand how to use a hammer though. Developers are not understanding how to use the hints properly which is causing you a personal headache. The hints aren’t bad, the programmers are untrained - the acknowledgement of this is the first step into a saner world.
As a static typing advocate I do find it funny how all the popular dynamic languages have slowly become statically typed. After decades of people saying it's not at all necessary and being so critical of statically typed languages.
When I was working on a fairly large TypeScript project it became the norm for dependencies to have type definitions in a relatively short space of time.
People adapt to the circumstances. A lot of Python uses are no longer about fast iteration on the REPL. Instead of that we are shipping Python to execute in clusters on very long running jobs or inside servers. It's not only about having to start all over after hours, it's simply that concurrent and distributed execution environments are hostile to interactive programming. Now you can't afford to wait for an exception and launch the debugger in postmortem. Or even if you do it's not very useful.
And now my personal opinion: If we are going the static typing way I would prefer simply to use Scala or similar instead of Python with types. Unfortunately in the same way that high performance languages like C attracts premature optimizers static types attract premature "abstracters" (C++ both). I also think that dynamic languages have the largest libraries for technical merit reasons. Being more "fluid" make them easier to mix. In the long term the ecosystem converges organically on certain interfaces between libraries.
And so here we are with the half baked approach of gradual typing and #type: ignore everywhere.
Here we are because:
* Types are expensive and dont tend to pay off on spikey/experimental/MVP code, most of which gets thrown away.
* Types are incredibly valuable on hardened production code.
* Most good production code started out spikey, experimental or as an MVP and transitioned.
And so here we are with gradual typing because "throwing away all the code and rewriting it to be "perfect" in another language" has been known for years to be a shitty way to build products.
Im mystified that more people here dont see that the value and cost of types is NOT binary ("they're good! theyre bad!") but exists on a continuum that is contingent on the status of the app and sometimes even the individual feature.
> Types are expensive and dont tend to pay off on spikey/experimental/MVP code, most of which gets thrown away.
I find I’ve spent so much time writing with typed code that I now find it harder to write POC code in dynamic languages because I use types to help reason about how I want to architect something.
Eg “this function should calculate x and return”, well if you already know what you want the function to do then you know what types you want. And if you don’t know what types you want then you haven’t actually decided what that function should do ahead of building it.
Now you might say “the point of experimental code is to figure out what you want functions to do”. But even if you’re writing an MVP, you should know what that each function should do by the time you’ve finished writing it. Because if you don’t know who to build a function then how do you even know that the runtime will execute it correctly?
Python doesn’t have “no types,” in fact it is strict about types. You just don’t have to waste time reading and writing them early on.
While a boon during prototyping, a project may need more structural support as the design solidifies, it grows, or a varied, growing team takes responsibility.
At some point those factors dominate, to the extent “may need” support approaches “must have.”
My point is if you don’t know what types you need, then you can’t be trusted to write the function to begin with. So you don’t actually save that much time in the end. typing out type names simply isn’t the time consuming part of prototyping.
But when it comes to refactoring, having type safety makes it very easy to use static analysis (typically the compiler) check for type-related bugs during that refactor.
I’ve spent a fair amount of years in a great many different PL paradigms and I’ve honestly never found loosely typed languages any fast for prototyping.
That all said, I will say that a lot of this also comes down to what you’re used to. If you’re used to thinking about data structures then your mind will go straight there when prototyping. If you’re not used to strictly typed languages, then you’ll find it a distraction.
Right after hello world you need a list of arguments or a dictionary of numbers to names. Types.
Writing map = {}, is a few times faster than map: Dictionary[int, str] = {}. Now multiply by ten instances. Oh wait, I’m going to change that to a tuple of pairs instead.
It takes me about three times longer to write equivalent Rust than Python, and sometimes it’s worth it.
Rust is slower to prototype than Python because Rust is a low level language. Not because it’s strictly typed. So that’s not really a fair comparison. For example, assembly doesn’t have any types at all and yet is slower to prototype than Rust.
Let’s take Visual Basic 6, for example. That was very quick to prototype in even with “option explicit” (basically forcing type declarations) defined. Quicker, even, than Python.
Typescript isn’t any slower to prototype in than vanilla JavaScript (bar setting up the build pipeline — man does JavaScript ecosystem really suck at DevEx!).
Writing map = {} only saves you a few keystrokes. And Unless you’re typing really slowly with one finger like an 80 year old using a keyboard for the first time, you’ll find the real input bottleneck isn’t how quickly you can type your data structures into code, but how quickly your brain can turn a product spec / Jira ticket into a mental abstraction.
> Oh wait, I’m going to change that to a tuple of pairs instead
And that’s exactly when you want the static analysis of a strict type system to jump in and say “hang on mate, you’ve forgotten to change these references too” ;)
Having worked on various code bases across a variety of different languages, the refactors that always scare me the most isn’t the large code bases, it’s the ones in Python or JavaScript because I don’t have a robust type system providing me with compile-time safety.
There’s an old adage that goes something like this: “don’t put off to runtime what can be done in compile time.”
As computers have gotten exponentially faster, we’ve seemed to have forgotten this rule. And to our own detriment.
Rust has many high-level constructs available as well as libraries ready and available if you stick to "python-like" things. Saving a "few keystrokes" is not what I described, it was specific: `: Dictionary[int, str]`, this is hard to remember, write, and read, and there's lots of punctuation. Many defs are even harder to compose.
Cementing that in early on is a big pre-optimization (ie waste) when it has a large likelyhood of being deleted. Refactors are not large at this point, and changes trivial to fix.
I've found the transition point where types are useful to start even within a few hundred lines of code, and I've found types are not that restrictive if at all, especially if the language started out typed. The rare case I need to discard types that is available usually, and a code smell your doing something wrong.
Even within a recent toy 1h python interview question having types would've saved me some issues and caught an error that wasn't obvious. Probably would've saved 10m in the interview.
Yep, depends on your memory context capacity.
For me I often don't feel any pain-points when working before about 1kloc (when doing JS), however if a project is above 500loc it's often a tad painful to resume it months later when I've started to forget why I used certain data-structures that aren't directly visible (adding types at that point is usually the best choice since it gives a refresher of the code at the same time as doing a soundness check).
The transition to where type hints become valuable or even necessary isnt about how many lines of code you have it is about how much you rely upon their correctness.
Type strictness also isnt binary. A program with lots of dicts that should be classes doesnt get much safer just because you wrote : dict[str, dict] everywhere.
> * Types are expensive and dont tend to pay off on spikey/experimental/MVP code, most of which gets thrown away.
This is what people say, but I don't think it's correct. What is correct is that say, ten to twenty years ago, all the statically typed languages had other unacceptable drawbacks and "types bad" became a shorthand for these issues.
I'm talking about C (nonstarter for obvious reasons), C++ (a huge mess, footguns, very difficult, presumably requires a cmake guy), Java (very restrictive, slow iteration and startups, etc.). Compared to those just using Python sounds decent.
Nowadays we have Go and Rust, both of which are pretty easy to iterate in (for different reasons).
> Nowadays we have Go and Rust, both of which are pretty easy to iterate in (for different reasons).
It's common for Rust to become very difficult to iterate in.
I think Java was the main one. C/C++ are (relatively) close to the metal, system-level languages with explicit memory management - and were tacitly accepted to be the "complicated" ones, with dynamic typing not really applicable at that level.
But Java was the high-level, GCed, application development language - and more importantly, it was the one dominating many university CS studies as an education language before python took that role. (Yeah, I'm grossly oversimplifying - sincere apologies to the functional crowd! :) )
The height of the "static typing sucks!" craze was more like a "The Java type system sucks!" craze...
For me it was more the “java can’t easily process strings” craze that made it impractical to use for scripts or small to medium projects.
Not to mention boilerplate BS.
Recently, Java has improved a lot on these fronts. Too bad it’s twenty-five years late.
> * Types are expensive and dont tend to pay off on spikey/experimental/MVP code, most of which gets thrown away.
Press "X" to doubt. Types help _a_ _lot_ by providing autocomplete, inspections, and helping with finding errors while you're typing.
This significantly improves the iteration speed, as you don't need to run the code to detect that you mistyped a varible somewhere.
Pycharm, pyflakes, et all can do most of these without written types.
The more interesting questions, like “should I use itertools or collections?” Autocomplete can’t help with.
In some fields throwing away and rewriting is the standard, and it works, more or less. I'm thinking about scientific/engineering software: prototype in Python or Matlab and convert to C or C++ for performance/deployment constraints. It happens frequently with compilers too. I think migrating languages is actually more successful than writing second versions.
The issue with moving the ship where it's passanger wants it to be makes it more difficult for new passengers to get on.
This is clearly seen with typescript and the movement for "just use JS".
Furthermore, with LLMs, it should be easier than ever to experiment in one language and use another language for production loads.
I don't think types are expensive for MVP code unless they're highly complicated (but why would you do that?) Primitives and interfaces are super easy to type and worth the extra couple seconds.
Software quality only pays off on the long time. For the short time, garbage is quick and gets the job done.
Also, in my experience, the long time for software arrives in a couple of weeks.
PHP is a great example of the convergence of interfaces. Now they have different “PSR” standards for all sorts of things. There is one for HTTP clients, formatting, cache interfaces, etc. As long as your library implements the spec, it will work with everything else and then library authors are free to experiment on the implementation and contribute huge changes to the entire ecosystem when they find a performance breakthrough.
Types seem like a “feature” of mature software. You don’t need to use them all the time, but for the people stuck on legacy systems, having the type system as a tool in their belt can help to reduce business complexity and risk as the platform continues to age because tooling can be built to assert and test code with fewer external dependencies.
Python is ubiquitous in ML, often you have no choice but to use it
[dead]
> slowly become statically typed
They don't. They become gradually typed which is a thing of it's own.
You can keep the advantages of dynamic languages, the ease of prototyping but also lock down stuff when you need to.
It is not a perfect union, generally the trade-off is that you can either not achieve the same safety level as in a purely statically typed language because you need to provide same escape hatches or you need a extremely complex type system to catch the expressiveness of the dynamic side. Most of the time it is a mixture of both.
Still, I think this is the way to go. Not dynamic typing won or static typing won but both a useful and having a language support both is a huge productivity boost.
> how all the popular dynamic languages have slowly become statically typed
Count the amount of `Any` / `unknown` / `cast` / `var::type` in those codebases, and you'll notice that they aren't particularly statically typed.
The types in dynamic languages are useful for checking validity in majority of the cases, but can easily be circumvented when the types become too complicated.
It is somewhat surprising that dynamic languages didn't go the pylint way, i.e. checking the codebase by auto-determined types (determined based on actual usage).
Julia (by default) does the latter, and its terrible. It makes it a) slow, because you have to do nonlocal inference through entire programs, b) impossible to type check generic library code where you have no actual usage, c) very hard to test that some code works generically, as opposed to just with these concrete types, and finally d) break whenever you have an Any anywhere in the code so the chain of type information is broken.
In the discussion of static vs dynamic typing solutions like typescript or annotated python were not really considered.
IMHO the idea of a complex and inference heavy type system that is mostly useless at runtime and compilation but focused on essentially interactive linting is relatively recent and its popularity is due to typescript success
I think that static typing proponents were thinking of something more along the lines of Haskell/OCaml/Java rather than a type-erased system a language where [1,2] > 0 is true because it is converted to "NaN" > "0"
OTH I only came to realize that I actually like duck typing in some situations when I tried to add type hints to one of my Python projects (and then removed them again because the actually important types consisted almost entirely of sum types, and what's the point of static typing if anything is a variant anyway).
E.g. when Python is used as a 'scripting language' instead of a 'programming language' (like for writing small command line tools that mainly process text), static typing often just gets in the way. For bigger projects where static typing makes sense I would pick a different language. Because tbh, even with type hints Python is a lousy programming language (but a fine scripting language).
> Because tbh, even with type hints Python is a lousy programming language (but a fine scripting language).
I'd be interested in seeing you expand on this, explaining the ways you feel Python doesn't make the cut for programming language while doing so for scripting.
The reason I say this is because, intuitively, I've felt this way for quite some time but I am unable to properly articulate why, other than "I don't want all my type errors to show up at runtime only!"
Learn how to use the tools to prevent that last paragraph.
Note1: Type hints are hints for the reader. If you cleverly discovered that your function is handling any type of data, hint that!
Note2: From my experience, in Java, i have NEVER seen a function that consumes explicitely an Object. In Java, you always name things. Maybe with parametric polymorphism, to capture complex typing patterns.
Note 3: unfortunately, you cannot subclass String, to capture the semantic of its content.
> Java, i have NEVER seen a function that consumes explicitely an Object
So you did not see any Java code from before version 5 (in 2004) then, because the language did not have generics for the first several years it was popular. And of course many were stuck working with older versions of the language (or variants like mobile Java) without generics for many years after that.
Exactly, I have never seen such codes [*].
Probably because the adoption of the generics has been absolutely massive in the last 20 years. And I expect the same thing to eventually happen with Typescript and [typed] Python.
[*]: nor have I seen EJB1 or even EJB2. Spring just stormed them, in the last 20 years.
An example of a function in Java that consumes a parameter of type Object is System.out.println(Object o)
Many such cases.
Sounds to be more of a symptom of the types of programs and functions you have written, rather than something inherent about types or Python. I've never encountered the type of gerry-mangled scenario you have described no matter how throwaway the code is.
If you like dynamic types have you considered using protocols? They are used precisely to type duck typed code.
> all the popular dynamic languages have slowly become statically typed
I’ve heard this before, but it’s not really true. Yes, maybe the majority of JavaScript code is now statically-typed, via Typescript. Some percentage of Python code is (I don’t know the numbers). But that’s about it.
Very few people are using static typing in Ruby, Lua, Clojure, Julia, etc.
Types become very useful when the code base reaches a certain level of sophistication and complexity. It makes sense that for a little script they provide little benefit but once you are working on a code base with 5+ engineers and no longer understand every part of it having some more strict guarantees and interfaces defined is very very helpful. Both for communicating to other devs as well as to simply eradicate a good chunk of possible errors that happen when interfaces are not clear.
Fair enough, apart from Ruby they’re all pretty niche.
OTOH I’m not arguing that most code should be dynamically-typed. Far from it. But I do think dynamic typing has its place and shouldn’t be rejected entirely.
Also, I would have preferred it if Python had concentrated on being the best language in that space, rather than trying to become a jack-of-all-trades.
You’re probably right. RedMonk [0] shows JavaScript and TypeScript separately and has the former well above the latter.
[0] https://redmonk.com/sogrady/2025/06/18/language-rankings-1-2...
Even if they're not written as TypeScript, there are usually add on definitions like "@types/prettier" and the like.
I disagree for Julia, but that probably depends on the definition of static typing.
For the average Julia package I would guess, that most types are statically known at compile time, because dynamic dispatch is detrimental for performance. I consider, that to be the definition of static typing.
That said, Julia functions seldomly use concrete types and are generic by default. So the function signatures often look similar to untyped Python, but in my opinion this is something entirely different.
At least in ruby theres mayor code bases using stripes sorbet and the official RBS standard for type hints. Notably its big code bases with large amounts of developers, fitting in with the trend most people in this discussion point to.
My last job was working at a company that is notorious for Ruby and even though I was mostly distant from it, there seemed to be a big appetite for Sorbet there.
The big difference between static typing in Python and Ruby is that Guido et al have embraced type hints, whereas Matz considers them to be (the Ruby equivalent of) “unpythonic”. Most of each language’s community follows their (ex-)BDFL’s lead.
PHP as well has become statically typed.
All the languages you name are niche languages compared to Python, JS (/ TS) and PHP. Whether you like it or not.
I think you're ignoring how for some of us, gradual typing, is a far better experience than languages with static types.
For example what I like about PHPStan (tacked on static analysis through comments), that it offers so much flexibility when defining type constraints. Can even specify the literal values a function accepts besides the base type. And subtyping of nested array structures (basically support for comfortably typing out the nested structure of a json the moment I decode it).
Not ignoring, I just didn't write an essay. In all that time working with TypeScript there was very little that I found to be gradually typed, it was either nothing or everything, hence my original comment. Sure some things might throw in a bunch of any/unknown types but those were very much the rarity and often some libraries were using incredibly complicated type definitions to make them as tight as possible.
Worked with python, typescript and now php, seems that phpstan allows this gradual typing, while typescript kinda forces you to start with strict in serious projects.
Coming from Java extreme verbosity, I just loved the freedom of python 20 years ago. Working with complex structures with mixed types was a breeze.
Yes, it was your responsibility to keep track of correctness, but that also taught me to write better code, and better tests.
Writing tests is harder work than writing the equvalent number of type hints though
Type hints and/or stronger typing in other languages are not good substitutes for testing. I sometimes worry that teams with strong preferences for strong typing have a false sense of security.
People write tests in statically typed languages too, it's just that there's a whole class of bugs that you don't have to test for.
Hints are not sufficient, you’ll need tests anyway. They somewhat overlap.
Writing and maintaining tests that just do type checking is madness.
Dynamic typing also gives tooling such as LSPs and linters a hard time figuring out completions/references lookup etc. Can't imagine how people work on moderate to big projects without type hints.
AI tab-complete & fast LSP implementations made typing easy. The tools changed, and people changed their minds.
JSON's interaction with types is still annoying. A deserialized JSON could be any type. I wish there was a standard python library that deserialized all JSON into dicts, with opinionated coercing of the other types. Yes, a custom normalizer is 10 lines of code. But, custom implementations run into the '15 competing standards' problem.
Actually, there should be a popular type-coercion library that deals with a bunch of these annoying scenarios. I'd evangelize it.
Type hints / gradual typing is crucially different from full static typing though.
It’s valid to say “you don’t need types for a script” and “you want types for a multi-million LOC codebase”.
Static typing used to be too rigid and annoying to the point of being counterproductive. After decades of improvement of parsers and IDEs they finally became usable for rapid development.
Everything goes in cycles. It has happened before and it will happen again. The softward industry is incredibly bad at remembering lessons once learned.
That's because many do small things that don't really need it, sure there are some people doing larger stuff and are happy to be the sole maintainer of a codebase or replace the language types with unit-test type checks.
And I think they can be correct for rejecting it, banging out a small useful project (preferably below 1000 loc) flows much faster if you just build code doing things rather than start annotating (that quickly can be come a mind-sinkhole of naming decisions that interrupts a building flow).
However, even less complex 500 loc+ programs without typing can become a pita to read after the fact and approaching 1kloc it can become a major headache to pick up again.
Basically, can't beat speed of going nude, but size+complexity is always an exponential factor in how hard continuing and/or resuming a project is.
Thing is, famous dynamic languages of the past, Common Lisp, BASIC, Clipper, FoxPro, all got type hints for a reason, then came a new generation of scripting languages made application languages, and everyone had to relearn why the fence was in the middle of the field.
I think both found middle ground. In Java you don’t need to define the type of variables within the method. In Python people have learned types in method arguments is a good thing.
> After decades of people saying
You have to admit that the size and complexity of the software we write has increased dramatically over the last few "decades". Looking back at MVC "web applications" I've created in the early 2000s, and comparing them to the giant workflows we deal with today... it's not hard to imagine how dynamic typing was/is ok to get started, but when things exceed one's "context", you type hints help.
I like static types but advocating for enforcing them in any situation is different. Adding them when you need (Python currently) seems a better strategy than forcing you to set them always (Typescript is in between as many times it can determine them).
Many years ago I felt Java typing could be overkill (some types could have been deduced from context and they were too long to write) so probably more an issue about the maturity of the tooling than anything else.
What I would need is a statically typed language that has first class primitives for working with untyped data ergonomically.
I do want to be able to write a dynamically typed function or subsystem during the development phase, and „harden” with types once I’m sure I got the structure down.
But the dynamic system should fit well into the language, and I should be able to easily and safely deal with untyped values and convert them to typed ones.
So… Typescript?
Yes, the sad part is that some people experienced early TypeScript that for some reason had the idea of forcing "class" constructs into a language where most people wasn't using or needing them (and still aren't).
Sometimes at about TypeScript 2.9 finally started adding constructs that made gradual typing of real-world JS code sane, but by then there was a stubborn perception of it being bad/bloated/Java-ish,etc despite maturing into something fairly great.
I like Typescript :)
The need for typing changed, when the way the language is used changed.
When JavaScript programs were a few hundred lines to add interactivity to some website type annotationd were pretty useless. Now the typical JavaScript project is far larger and far more complex. The same goes for python.
dynamically-typed languages were typically created for scripting tasks - but ended up going viral (in part due to d-typing), the community stretched the language to its limits and pushed it into markets it wasn't designed/thought for (embedded python, server-side js, distributed teams, dev outsourcing etc).
personally i like the dev-sidecar approach to typing that Python and JS (via TS) have taken to mitigate the issue.
Javascript is no longer was just scripting. Very large and complex billion dollar apps were being written in pure Javascript. It grew up.
I guess Python is next.
Next stop is to agree that JSON is really NOT the semantic data exchange serialization for this "properly typed" world.
Then what is?
Everybody knows the limitations of JSON. Don't state the obvious problem without stating a proposed solution.
The RDF structure is a graph of typed instances of typed objects, serializable as text.
Exchanging RDF, more precisely its [more readable] "RDF/turtle" variant, is probably what will eventually come to the market somehow.
Each object of a RDF structure has a global unique identifier, is typed, maintains typed links with other objects, have typed values.
For an example of RDF being exchanged between a server and a client, you can test
https://search.datao.net/beta/?q=barack%20obama
Open your javascript console, and hover the results on the left hand side of the page with your mouse. The console will display which RDF message triggered the viz in the center of the page.
Update: you may want to FIRST select the facet "DBPedia" at the top of the page, for more meaningful messages exchanged.
Update2: the console does not do syntax higlighting, so here is the highlighted RDF https://datao.net/ttl.jpg linked to the 1st item of " https://search.datao.net/beta/?q=films%20about%20barack%20ob... "
That's a circular argument. What serialization format would you recommend? JSON?
Turtle directly.
JSON forces you to fit your graph of data into a tree structure, that is poorly capturing the cardinalities of the original graph.
Plus of course, the concept of object type is not existing in JSON.
Thank you, I did not realize that RDF has its own serialization format. I'm reading about it now.
I think that the practically available type checkers evolved to a point where many of the common idioms can be expressed with little effort.
If one thinks back to some of the early statically typed languages, you'd have a huge rift: You either have this entirely weird world of Caml and Haskell (which can express most of what python type hints have, and could since many years), and something like C, in which types are merely some compiler hints tbh. Early Java may have been a slight improvement, but eh.
Now, especially with decent union types, you can express a lot of idioms of dynamic code easily. So it's a fairly painless way to get type completion in an editor, so one does that.
Trends change. There is still no hard evidence that static types are net positive outside of performance.
Huh. It's almost like these people didn't know what they were talking about. How strange.
Well, we do coalesce on certain things... some static type languages are dropping type requirements (Java and `var` in certain places) :D
There's no dropping of type requirements in Java, `var` only saves typing.
When you use `var`, everything is as statically typed as before, you just don't need to spell out the type when the compiler can infer it. So you can't (for example) say `var x = null` because `null` doesn't provide enough type information for the compiler to infer what's the type of `x`.
> `var` only saves typing.
this is a lovely double entendre
var does absolutely nothing to make Java a less strictly typed language. There is absolutely no dropping of the requirement that each variable has a type which is known at compile time.
Automatic type inference and dynamic typing are totally different things.
I have not written a line of Java in at least a decade, but does Java not have any 'true' dynamic typing like C# does? Truth be told, the 'dynamic' keyword in C# should only be used in the most niché of circumstances. Typically, only practitioners of Dark Magic use the dynamic type. For the untrained, it often leads one down the path of hatred, guilt, and shame. For example:
dynamic x = "Forces of Darkness, grant me power";
Console.WriteLine(x.Length); // Dark forces flow through the CLR
x = 5;
Console.WriteLine(x.Length); // Runtime error: CLR consumed by darkness.
C# also has the statically typed 'object' type which all types inherit from, but that is not technically a true instance of dynamic typing.
Same nonsense repeated over and over again... There aren't dynamic languages. It's not a thing. The static types aren't what you think they are... You just don't know what you are saying and your conclusion is just a word salad.
What happened to Python is that it used to be a "cool" language, whose community liked to make fun of Java for their obsession with red-taping, which included the love for specifying unnecessary restrictions everywhere. Well, just like you'd expect from a poorly functioning government office.
But then everyone wanted to be cool, and Python was adopted by the programming analogue of the government bureaucrats: large corporations which treat programming as a bureaucratic mill. They don't want fun or creativity or one-of bespoke solutions. They want an industrial process that works on as large a scale as possible, to employ thousands of worst quality programmers, but still reliably produce slop.
And incrementally, Python was made into Java. Because, really, Java is great for producing slop on an industrial scale. But the "cool" factor was important to attract talent because there used to be a shortage, so, now you have Python that was remade to be a Java. People who didn't enjoy Java left Python over a decade ago. So that Python today has nothing in common with what it was when it was "cool". It's still a worse Java than Java, but people don't like to admit defeat, and... well, there's also the sunk cost fallacy: so much effort was already spent at making Python into a Java, that it seems like a good idea to waste even more effort to try to make it a better Java.
Yeah, this is the lens through which I view it. It's a sort of colonization that happens, when corporations realize a language is fit for plunder. They start funding it, then they want their people on the standards boards, then suddenly the direction of the language is matched very nicely to their product roadmap. Meanwhile, all the people who used to make the language what it was are bought or pushed out, and the community becomes something else entirely.
I love typing in Python. I learnt programming with C++ and OOPs. It was freeing when I took up Python to note care about types, but I have come to enjoy types as I got older.
But, boy have we gone overboard with this now? The modern libraries seem to be creating types for the sake of them. I am drowning in nested types that seem to never reach native types. The pain is code examples of the libraries don’t even show them.
Like copy paste an OpenAI example and see if LSP is happy for example. Now I have gotten in this situation where I am mentally avoiding type errors of some libraries and edging into wishing Pydantic et al never happened.
My love for python was critically hurt when I learned about typing.TYPE_CHECKING.
For those unaware, due to the dynamic nature of Python, you declare a variable type like this
foo: Type
This might look like Typescript, but it isn't because "Type" is actually an object. In python classes and functions are first-class objects that you can pass around and assign to variables.The obvious problem of this is that you can only use as a type an object that in "normal python" would be available in the scope of that line, which means that you can't do this:
def foo() -> Bar:
return Bar()
class Bar:
pass
Because "Bar" is defined AFTER foo() it isn't in the scope when foo() is declared. To get around this you use this weird string-like syntax: def foo() -> "Bar":
return Bar()
This already looks ugly enough that should make Pythonists ask "Python... what are you doing?" but it gets worse.If you have a cyclic reference between two files, something that works out of the box in statically typed languages like Java, and that works in Python when you aren't using type hints because every object is the same "type" until it quacks like a duck, that isn't going to work if you try to use type hints in python because you're going to end up with a cyclic import. More specifically, you don't need cyclic imports in Python normally because you don't need the types, but you HAVE to import the types to add type hints, which introduces cyclic imports JUST to add type hints. To get around this, the solution is to use this monstrosity:
if typing.TYPE_CHECKING:
import Foo from foo
And that's code that only "runs" when the static type check is statically checking the types.Nobody wants Python 4 but this was such an incredibly convoluted way to add this feature, specially when you consider that it means every module now "over-imports" just to add type hints that they previously didn't have to.
Every time I see it makes me think that if type checks are so important maybe we shouldn't be programming Python to begin with.
There's actually another issue with ForwardRefs. They don't work in the REPL. So this will work when run as a module:
def foo() -> "Bar":
return Bar()
But will throw an error if copy pasted into a REPL.However, all of these issues should be fixed in 3.14 with PEP649 and PEP749:
> At compile time, if the definition of an object includes annotations, the Python compiler will write the expressions computing the annotations into its own function. When run, the function will return the annotations dict. The Python compiler then stores a reference to this function in __annotate__ on the object.
> This mechanism delays the evaluation of annotations expressions until the annotations are examined, which solves many circular reference problems.
It doesn't throw error in the REPL though. Surely you meant to share some other example?
Please ignore my first assertion that the behavior between REPL and module is different.
This would have been the case if the semantics of the original PEP649 spec had been implemented. But instead, PEP749 ensures that it is not [0]. My bad.
> that isn't going to work if you try to use type hints in python because you're going to end up with a cyclic import. More specifically, you don't need cyclic imports in Python normally because you don't need the types, but you HAVE to import the types to add type hints, which introduces cyclic imports JUST to add type hints.
Yes, `typing.TYPE_CHECKING` is there so that you can conditionally avoid imports that are only needed for type annotations. And yes, importing modules can have side effects and performance implications. And yes, I agree it's ugly as sin.
But Python does in fact allow for cyclic imports — as long as you're importing the modules themselves, rather than importing names `from` those modules. (By the way, the syntax is the other way around: `from ... import ...`.)
This is trivial to solve by simply not having circular imports. Place the types in one file and the usage of it in others.
This has many benefits, like forcing you to think about the dependencies and layers of your architecture. Here is a good read about why, from F# that has the same limitation https://fsharpforfunandprofit.com/posts/cyclic-dependencies/
As others already mentioned, importing __annotations__ also works.
If the type is a class with methods, then this method doesn't work, though adding intermediate interface classes (possibly with Generic types) might help in most cases. Python static type system isn't quite the same level as F#.
> Well, these complaints are unfounded.
"You're holding it wrong." I've also coded quite a bit of OCaml and it had the same limitation (which is where F# picked it up in the first place), and while the issue can be worked around, it still seemed to creep up at times. Rust, also with some virtual OCaml ancestry, went completely the opposite way.
My view is that while in principle it's a nice property that you can read and and understand a piece of code by starting from the top and going to the bottom (and a REPL is going to do exactly that), in practice it's not the ultimate nice property to uphold.
> If the type is a class with methods, then this method doesn't work
Use typing.Self
I meant if you have two classes that need to refer to each other. But good pointer anyway, I hadn't noticed it, thanks!
I ran into some code recently where this pattern caused me so much headache - class A has an attribute which is an instance of class B, and class B has a "parent" attribute (which points to the instance of class A that class B is an attribute of):
class Foo:
def __init__(self, bar):
self.bar = bar
class Bar:
def __init__(self, foo):
self.foo = foo
Obviously both called into each other to do $THINGS... Pure madness.So my suggestion: Try not to have interdependent classes :D
Well, at times having a parent pointer is rather useful! E.g. a callback registration will be able to unregister itself from everywhere where it has been registered to, upon request. (One would want to use weak references in this case.)
Fair point!
Maybe I am just a bit burned by this particular example I ran into (where this pattern should IMO not have been used).
> If you have a cyclic reference between two files,
Don't have cyclic references between two files.
It makes testing very difficult, because in order to test something in one file, you need to import the other one, even though it has nothing to do with the test.
It makes the code more difficult to read, because you're importing these two files in places where you only need one of them, and it's not immediately clear why you're importing the second one. And it's not very satisfying to learn that you you're importing the second one not because you "need" it but because the circular import forces you to do so.
Every single time you have cyclic references, what you really have are two pieces of code that rely on a third piece of code, so take that third piece, separate it out, and have the first two pieces of code depend on the third piece.
Now things can be tested, imports can be made sanely, and life is much better.
Using the typical "Rust-killer" example: if you have a linked list where the List in list.py returns a Node type and Node in node.py takes a List in its constructor, you already have a cyclic reference.
Agreed that this "hack" is very ugly!
On the other hand, I tend to take it as a hint that I should look at my module structure, and see if I can avoid the cyclic import (even if before adding type hints there was no error, there still already was a "semantic dependency"...)
You're actually missing the benefit of this. It's actually a feature.
With python, because types are part of python itself, they can thus be programmable. You can create a function that takes in a typehint and returns a new typehint. This is legal python. For example below I create a function that dynamically returns a type that restricts a Dictionary to have a specific key and value.
from typing import TypedDict
def make_typed_dict(name: str, required_key: str, value_type: type):
return TypedDict(name, {required_key: value_type, id: int})
# Example
UserDict = make_typed_dict("UserDict", "username", str)
def foo(data: UserDict):
print(data["username"])
With this power in theory you can create programs where types essentially can "prove" your program correct, and in theory eliminate unit tests. Languages like idris specialize in this. But it's not just rare/specialized languages that do this. Typescript, believe it or not, has programmable types that are so powerful that writing functions that return types like the one above are Actually VERY common place. I was a bit late to the game to typescript but I was shocked to see that it was taking cutting edge stuff from the typing world and making it popular among users.In practice, using types to prove programs to be valid in place of testing is actually a bit too tedious compared with tests so people don't go overboard with it. It is a much more safer route then testing, but much harder. Additionally as of now, the thing with python is that it really depends on how powerful the typechecker is on whether or not it can enforce and execute type level functions. It's certainly possible, it's just nobody has done it yet.
I'd go further than this actually. Python is actually a potentially more powerfully typed language than TS. In TS, types are basically another language tacked onto javascript. Both languages are totally different and the typing language is very very limited.
The thing with python is that the types and the language ARE the SAME thing. They live in the same universe. You complained about this, but there's a lot of power in that because basically types become turing complete and you can create a type that does anything including proving your whole program correct.
Like I said that power depends on the typechecker. Someone needs to create a typechecker that can recognize type level functions and so far it hasn't happened yet. But if you want to play with a language that does this, I believe that language is Idris.
That's not a benefit. That's a monstrosity.
And, as you heavily imply in your post, type checkers won't be able to cope with it, eliminating one if the main benefits of type hints. Neither will IDEs / language servers, eliminating the other main benefit.
>And, as you heavily imply in your post, type checkers won't be able to cope with it
I implied no such thing. literally said there's a language that already does this. Typescript. IDE's cope with it just fine.
>That's not a benefit. That's a monstrosity.
So typescript is a monstrosity? Is that why most of the world who uses JS in node or the frontend has moved to TS? Think about it.
I don't believe Typescript (nor Idris) type systems work like you describe, though? Types aren't programmable with code like that (in the same universe, as you say) and TS is structurally typed, with type erasure (ie types are not available at runtime).
I am not that deeply familiar with Python typings development but it sounds fundamentally different to the languages you compare to.
Typescript types (and Idris) are Turing complete. You can actually get typescript types to run doom.
https://www.youtube.com/watch?v=0mCsluv5FXA&t
Idris on the other hand is SPECIFICALLY designed so types and the program live in the same language. See the documentation intro: https://www.idris-lang.org/pages/example.html
THe powerful thing about these languages is that they can prove your program correct. For testing you can never verify your program to be correct.
Testing is a statistical sampling technique. To verify a program as correct via tests you have to test every possible input and output combination of your program, which is impractical. So instead people write tests for a subset of the possibilities which ONLY verifies the program as correct for that subset. Think about it. If you have a function:
def add(x: int, y: int) -> int
How would you verify this program is 100% correct? You have to test every possible combination of x, y and add(x, y). But instead you test like 3 or 4 possibilities in your unit tests and this helps with the overall safety of the program because of statistical sampling. If a small sample of the logic is correct, it says something about the entire population of the logic..Types on the other hand prove your program correct.
def add(x: int, y: int) -> int:
return x + y
If the above is type checked, your program is proven correct for ALL possible types. If those types are made more advanced via being programmable, then it becomes possible for type checking to prove your ENTIRE program correct.Imagine:
def add<A: addable < 4, B: addable < 4>(x: A, y: B) -> A + B:
return x + y
With a type checker that can analyze the above you can create a add function that at most can take an int that is < 4 and return an int that is < 8. Thereby verifying even more correctness of your addition function.Python on the other hand doesn't really have type checking. It has type hints. Those type hints can de defined in the same language space as python. So a type checker must read python to a limited extent in order to get the types. Python at the same time can also read those same types. It's just that python doesn't do any type checking with the types while the type checker doesn't do anything with the python code other than typecheck it.
Right now though, for most typecheckers, if you create a function in python that returns a typehint, the typechecker is not powerful enough to execute that function to find the final type. But this can certainly be done if there was a will because Idris has already done this.
The syntax is a monstrosity. You can also extract a proven OCaml program from Coq and Coq has a beautiful syntax.
If you insist on the same language for specifying types, some Lisp variants do that with a much nicer syntax.
Python people have been indoctrinated since ctypes that a monstrous type syntax is normal and they reject anything else. In fact Python type hints are basically stuck on the ctypes level syntax wise.
That's just a sugar thing. Yeah it can get a bit more verbose.
That's horrible. Nobody needs imperative metaprogramming for type hints. In fact, it would be absolute insanity for a typechecker to check this because it would mean opening a file in VS code = executing arbitrary python code. What stops me from deleting $HOME inside make_typed_dict?
TypeScript solves this with its own syntax that never gets executed by an interpreter because types are striped when TS is compiled to JS.
>VS code = executing arbitrary python code. What stops me from deleting $HOME inside make_typed_dict?
Easy make IO calls illegal in the type checker. The type checker of course needs to execute code in a sandbox. It won't be the full python language. Idris ALREADY does this.
Are there really productive projects which rely on types as a proofing system? I've always thought it added too much complexity to the code, but I'd love to see it working well somewhere. I love the idea of correctness by design.
No too my knowledge nothing is strict about a proofing system because like I said it becomes hard to do. It could be useful for ultra safe software but for most cases the complexity isn't worth it.
But that doesn't mean it's not useful to have this capability as part of your typesystem. It just doesn't need to be fully utilized.
You don't need to program a type that proves everything correct. You can program and make sure aspects of the program are MORE correct than just plain old types. typescript is a language that does this and it is very common to find types in typescript that are more "proofy" than regular types in other languages.
See here: https://www.hacklewayne.com/dependent-types-in-typescript-se...
Typescript does this. Above there's a type that's only a couple of lines long that proves a string reversal function reverses a string. I think even going that deep is overkill but you can define things like Objects that must contain a key of a specific string where the value is either a string or a number. And then you can create a function that dynamically specifies the value of the key in TS.
I think TS is a good example of a language that practically uses proof based types. The syntax is terrible enough that it prevents people from going overboard with it and the result is the most practical application of proof based typing that I seen. What typescript tells us that proof based typing need only be sprinkled throughout your code, it shouldn't take it all over.
I want to say it (or something similar at least) was originally addressed by from __future__ import annotations back in Python 3.7/3.8 or thereabouts? I definitely remember having to use stringified types a while back but I haven't needed to for quite a while now.
Yes, annotations allows you to use the declared types as they are, no strings.
It turns them into thunks (formerly strings) automatically, an important detail if you're inspecting annotations at run time because the performance hit of resolving the actual type can be significant.
TIL, thanks! It looks like 3.14 is also changing it so that all evaluations are lazy.
At last, Pi-thon.
from __future__ import annotations
> But, boy have we gone overboard with this now? The modern libraries seem to be creating types for the sake of them. I am drowning in nested types that seem to never reach native types.
Thought you were talking about TypeScript for a moment there.
Except that typescript structural typing and features make it much easier to swim.
Also python is far less aggressive with lint warnings so it is much easier to make mistakes
I learned C++ before learning python as well and python felt like a breath of fresh air.
At first I thought it was because of the lack of types. But in actuality the lack of types was a detriment for python. It was an illusion. The reason why python felt so much better was because it had clear error messages and a clear path to find errors and bugs.
In C++ memory leaks and seg faults are always hidden from view so EVEN though C++ is statically typed, it's actually practically less safe then python and much more harder to debug.
The whole python and ruby thing exploding in popularity back in the day was a trick. It was an illusion. We didn't like it more because of the lack of typing. These languages were embraced because they weren't C or C++.
It took a decade for people to realize this with type hints and typescript. This was a huge technical debate and now all those people were against types are proven utterly wrong.
> It was an illusion. We didn't like it more because of the lack of typing. These languages were embraced because they weren't C or C++.
It's an illusion only you once had. Java (a language that is not C or C++) got mainstream way before Python.
Java on the other hand had the most verbose syntax known to man, especially those early versions of it. Nowadays it’s getting more tolerable.
I don't understand, the parent says that not being C/C++ was a strong point and you give an counter example of a successful language that is not C/C++
modern C++ is great, to be honest.