Python is an open sourced, interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together.
Python is simple and straightforward enough where we can become productive pretty rapidly. Now a lot of libraries have been written for Python, making it hard to change to a better language. it’s easy to learn, and currently it has the best AI/Statistics tools
Sometimes we think of python as the “duct tape” of the software development toolkit. We can get all kinds of high-performance tools and libraries written in C/C++ with python wrappers that we can then easily “duct tape” together in python to create your own programs.
Even though python itself is a bit slow, it’s not such a big deal because that’s only the very top layer of the whole program.
For example, Numpy is essentially a Python wrapper around BLAS (the Basic Linear Algebra Subprograms) plus higher-level APIs built on top and higher-level API code in Numpy is mostly written in C.
Another example is, SciPy is a wrapper for LAPACK (the Linear Algebra Package) plus higher-level APIs.
In case of you are building production code, once you tool it out with logging, thread safety, metrics, etc. python ends up with a fair amount of boilerplate and it becomes more about expressiveness because python programs are typically far shorter and more idiomatic than other languages.
We know, there are certainly places where performance matters, but most languages of this class — including Python — allows you to use extensions in compiled languages for when performance matters.
In scientific computing, the N for the top-level script can be very very low (as low as 1) quite frequently. Which is why Python is so popular there. The main stuff we use python for:
- Automated Test rigs/jigs — limited by the speed of the test equipment / process time (EG soak test) / speed of the production line
- Glue — taking data from X and transforming it into Y, if we have to run a script once a month to convert some data file into a spreadsheet it takes me longer to find it than it takes to run it — and saves me a load of manual work
- Data analysis — if we run a python script that decodes/analyses a large wireshark capture file the speed limit is probably the hard drive. But who cares anyway because the analysis saves us hours of manual work and we can be doing other stuff while it’s thinking
- Logging — long term data logging for debugging / monitoring, the code can be as slow as hell because all it has to do is capture data at the rate we need and write it to a disk
- Housekeeping — scripts that run in the background and/or as cron jobs we don’t care if they’re not optimised as they are only running for maybe a few seconds per day.
Machine learning applications are not slow in python. you’re offloading your work to highly optimized compiled libraries. Python is a glue language that makes it easy to chain library calls.
In fact, in the vast majority of cases, speed doesn’t matter:
- The absolute difference between the fastest and slowest execution time, given the current technology (both software and hardware), is minuscule. The relative difference might be large — maybe your fine-tuned C++ code executes in 30 us, whereas the Python code doing the same thing takes 100x time — but the absolute difference is still below 3 ms. Humans can’t perceive a difference in 3 ms.
- This is further augmented by the fact that the vast majority of current software is written for the web, and the web is slow. When a user has to wait for 250 ms for a request round trip, 3 ms becomes irrelevant even relatively.
- Finally, whereas the speed of execution rarely matters, the speed of writing code — and the ease of maintenance — matters almost always, and that tends to be better when coding in higher-level languages.
Python is ideal for IO-bound routines, where speed is irrelevant. It’s also fast enough for 95% of projects.
Python is great at concurrency! Achieving parallelism is not the only reason to do concurrency, the main one is doing I/O stuff and python makes it super easy to do that.
Python is seldom the best language to use for anything, but it is often the second best, which makes it a good general purpose language for people to know.
Python’s massive and easily accessible library set (thanks PIP!) solves a lot of the hard problems for you.
Python provides a good balance of strictness and usability: it is easy to write a maintainable program that does what it is supposed to do.