Python Performance: Why 'if not list' is 2x Faster Than Using len()
Python Performance: Why 'if not list' is 2x Faster Than Using len()

Python Performance: Why 'if not list' is 2x Faster Than Using len()

Python Performance: Why 'if not list' is 2x Faster Than Using len()
Python Performance: Why 'if not list' is 2x Faster Than Using len()
I think the explicitness of checking length is worth the performance cost. If ur writing code for speed ur not using python.
I'd argue that if it's strict explicitness you want, python is the wrong language. if not var
is a standard pattern in python. You would be making your code slower for no good reason.
I never understood that argument. If you can be sure the type is a collection (and this you always should) not list
is so moch easier to read and understood than the length check.
How many elements in that list? Ah, it's not list. It's list, of course, we checked. But it's not.
Compact does not mean easier to understand. They are different tests. The point is, do not make code less readable for speed unless speed matters. I am not going to argue which is more readable in any specific case. That is up to the developer.
Is it? Why introduce an additional conversion from not list means empty list that u have to hold in your head. I want to check length I check length I would argue is easyer to comprehend.
Why? not x
means x is None or len(x) == 0
for lists. len(x) == 0
will raise an exception if x
is None. In most cases, the distinction between None
and []
isn't important, and if it is, I'd expect separate checks for those (again, for explicitness) since you'd presumably handle each case differently.
In short:
None
and []
is important, have separate checksnot x
should be your default, since that way it's a common pattern for all typesIn complex cases where speed is less important than maintainability, I tend to agree.
In this case, a simple comment would suffice. And in fact nothing at all would be okay for any half-competent Python coder, as testing if lists are empty with if not
is super-standard.
Another use case is to reduce maintenance cost
One of the principles of the Pythonic style is: simple is better than complex.
But is it when you consider ambiguity and expressiveness too?
len(mylist)
tells me it's definitely a list and not null or whatever. It also tells me the intent of whoever writes the code was checking length.
If it requires a long article to explain, I'd certainly be hesitant to use and prefer. The question is, is it surprising or intuitive that the truthy becomes a length check. What do you want to express with your code. And who will read it, and what expectations and requirements do you want to require of them.
For sequence type objects, such as lists, their truth value is False if they are empty.
len(mylist) tells me itβs definitely a list and not null or whatever
Do you know what else can tell you it's a list? Type hinting.
If I care about the distinction between None and an empty list, I'll make separate checks. len(None)
raises an exception, and most of the time that's not what I want when checking whether there's something in the list, so not x
is generally preferred, especially when type hinting is used consistently enough to be reasonably sure I'm actually getting a list or None
. If I get something else, that means things got really broken and they'll likely get an exception alter (i.e. when doing anything list-like).
For sequence type objects, such as lists, their truth value is False if they are empty.
That's generally exactly what I want. len(x) == 0
doesn't tell you it's a list, it just tells you it has __len__
. So that could be a dict
, list
, or a number of other types that have a length, but aren't lists.
Don't rely on checks like that to tell you what type a thing is.
It's just how pythonic code is written. All python developers know that not
and bool
can indicate that variable is either empty or null. They will also use the in
operator and other python specific syntax.
foo = "potatoes!"
len(foo)
will give you the number of characters in the string. It's not a surefire way to find a list at all!
Same with dictionaries, iterators (will consume the iterator!), and anything else that has a length.
I've actually had the string instead of list issue a number of times, because sometimes things get off and it's hard to track down where things went wrong.
For this reason, I think type hints are the way to go. Use them consistently w/ a static analysis tool like pyright
and you'll catch a lot of these issues before running into cryptic error messages.
I don't know about others ... but I'm not using Python for execution speed.
Typically the biggest problem in a program is not 100 million calls of len(x) == 0
. If it was, the interpreter could just translate that expression during parsing.
I mean, nobody uses Python for execution speed precisely because it is so slow.
Even so, not x
is a pretty nice-to-read pattern, and it's nice that it's faster than the less nice len(x) == 0
. I generally do not care to distinguish whether a list is None
or empty, I just want to know if there's something there. If I care, then I'll typically separate those checks anyway: if x is None: ...
, if len(x) == 0: ...
.
The interpreter can't make the replacement until it's about to execute the line as __bool__
and __len__
are both (Python's equivalent of) virtual functions, so it's got to know the runtime type to know if the substitution is valid. Once it's that late, it'll often be faster to execute the requested function than work out if it could execute something faster instead. Even with type hints, it's possible that a subclass with overridden methods could be passed in, so it's not safe to do anything until the real runtime type is known.
Once there's a JIT involved, there's an opportunity to detect the common types passed to a function and call specialised implementations, but I don't think Python's JIT is clever enough for this. LuaJIT definitely does this kind of optimisation, though.
Hm... I'll admit I wasn't awkward of the .__len__
function. However, does this mean it's possible to write a len(x) == 0
that's diverges from not x
?
If not, then the substitution is still valid (and not
presumably also considers the same fundamental. If so, that's kind of silly.
EDIT: I missed the part of your comment about .__bool__
... so yeah in theory you could have something where these two operations are not equivalent.
Arguably, you could just say that's pathological and invalid. Then, still have an optimized path to prefer not .__bool__()
if .__len__() == 0
is the comparison you'd be making. Even with the extra interpreter check during evaluation, that would quite possibly be faster if the overhead is truly high.
EDIT 2: you'd probably need a little bit more overhead than a straight substitution anyways because you need to maintain the semantic of "if this thing is None
it's not valid if the syntax was originally len(x)
."
This. I rarely boot up Python for the tasks I need to do, and if they are, they are one of the following:
Assuming an equivalent package is produced, what's the maintenance cost (factoring in coder availability) difference between the Python vs faster language implementations?
^^ therein lies the rub
Reminds of the expression, premature optimization is the root of all evil
if not swimming in funding, might be a darwinic move to choose the faster language and have to code everything yourself from scratch
Makes perfect sense. If you're checking if a collection is empty you don't need to know its exact size. Getting the size can be very inefficient in collections like linked lists or trees, if you have to follow all nodes. To check if it's empty, all you need fo know if at least one item exists. If one does, there's no point counting the rest.
People who don't understand the difference will probably not understand the difference between passing a list and passing an literator/generator to any()
.
While the 2nd approach is not wrong, the first method is considered more Pythonic. Many people donβt agree, but Iβve already put forward my points in a previous article on that debate.
Does Pythonic mean best practice in the python community or "good python"?
If "many people don't agree", how can they claim it to be "pythonic"? Isn't that contradictory?
"Many" isn't the same as "most," though I don't think there's any way to know what "most" is.
But here's my reason for agreeing w/ the OP:
not x
checks both None
and emptiness, which is usually desiredlen(x) == 0
will raise an exception if x
is nullIt's nice that it's slightly faster, though performance has absolutely nothing to do w/ my preference, though it does have something to do with my preference for avoiding exceptions for non-exceptional cases.
lol this whole article and this thread is a bunch of bikeshedding I haven't had the privilege of enduring in some weeks.
One more ridiculous fact about the Python to know Thank you for sharing.