PyCon Israel 2023 - Conference

Let me take you on a debugging story - where you'll learn why understanding how your web application runs in production can make you a better full-stack developer.

Have you ever faced the challenge of debugging a web application that runs differently in production than in your local environment?

In this talk, we'll explore the importance of gaining a deeper understanding of Gunicorn and NGINX, two critical components of the modern web stack, to become a better developer. We'll dive into the power of greenlets, exploring how they offer a lightweight approach to concurrency management and we'll explore how all of this work together to handle HTTP requests, manage concurrency, and distribute workload across multiple worker processes.

We'll wrap up the talk by exploring some Python tips to help us write thread-aware code.

Lets get to know Python in depth by exploring the boundaries of its flexibility... boldly going where no man should ever go, especially not in production. Content Warning: we might break the record of wtfs/minute in an official conference talk.

Have you ever used cffi to mutate an immutable str? Used fuckit('module') to import a module that tends to throw random fits of anger (and exceptions)? Used try: 1/0; except: import inspect to change a function's behavior based on who called it? Used an import hook to extend Python's syntax with your own syntactic constructs? Implemented a cache for your functions that parametrizes on the last modified time of the module they and their arguments are defined in and every module it depends on?

No? Good.

But someone had to check the grass isn't greener in hell, so you wouldn't be tempted to, right? And honestly, I've seen the last two running in production sorry i really gotta go ok thanks byeee dodges

Discover the untapped potential of Agent-Based Modeling (ABM) in various fields like finance, social interactions, and cybersecurity. Learn to implement ABM with Python and Mesa, and (yes!) see GPT integrated within ABM because everyone wants GPT.

With recent advances in AI, novel methods like generative AI and large language models have captured significant attention. However, Agent-Based Modeling (ABM) offers unique advantages and versatility for various applications such as financial modeling, social interactions, and cybersecurity. In this talk, I aim to demonstrate the potential of ABM for fast experimentation and hypothesis testing by modeling the world through isolated, manageable components. By breaking down complex systems into simpler elements, ABM allows for easier exploration and understanding of the underlying dynamics. I will showcase various use cases and provide Python code snippets using the Mesa package to illustrate the practical implementation of ABM. Join me in discovering the untapped potential of Agent-Based Modeling and learn how to implement this powerful approach in your projects with the help of Python and the Mesa package. As an exciting bonus, I'll also demonstrate how to integrate GPT within ABM, because everyone wants a piece of GPT in their projects!

This talk will delve into the AST module's capabilities and limitations, covering its class hierarchy, general structure, and supported functions. Attendees will be presented with examples of cool code manipulations based on AST.

The AST (Abstract Syntax Tree) module in Python can be used to analyze code, alert it, and even generate new code. We can use programming to change the AST representation of python, resulting in code that looks and behaves differently than the original. This is a really cool thing to do.

In this talk we will explore the AST module capabilities, and its lack of capabilities. We will learn the general structure of ASTmodule, its class hierarchy and its supported functions. Our focus will be on acquiring the skill to recreate the initial code through its AST representation. It seems that even though transforming python code into an AST is a simple task, the inverse process of transforming AST into code fails to replicate the original code.

We will see some examples of fun code-manipulations. First, we will show how to automatically add meaningful logs to your code. Second, we will experiment with code reordering and last, we will explore ideas to manipulate code for mutant testing purposes.

You can do many cool things with affordable small microprocessors and ONLY with code (No electronic knowledge!), from network gizmos to cybersecurity tools.

ESP32 is a 4$ only microprocessor, it is a NON-Arduino microprocessor with built-in WiFi and Bluetooth, and you can work with it without electronic knowledge. Anyone with only expertise in Python can program and build a networking chip for monitoring your home network or a security infiltration tool that can hack WiFi networks or anything else – the sky is the limit! In this lecture, I will explain ESP32 and show a code and demo of the practical and fun projects you can build with it.

Datetime in your DB is easy when your app is local, but what if one client writes in USA and another reads from France? We’ll see some challenges in adding time constraints to a global app and suggest the aviation industry methodology to solve them.

Any complex Django-based app will include many models that use the DateTime field, and the clients use those fields.

Assuming all our clients will always be in the same timezone is wrong for a globally used app. Pythons DateTime and pytz packages can help you localize the information, but what is the “right localization”? And more importantly, how can we assume where to localize from?

When designing a flow that handles DateTime fields in server-client data exchange, timezone conversions are the main issue.

Another challenge is what can we assume about the DateTime data that comes from the client when they send a PUT request with a DateTime field? What should the response assume? What if the client sends a GET request without any additional data?

In this talk, we will address those issues by demonstrating a simple “Timed Assignment App” that can be used globally. I’ll show the thought process that might go through your mind when designing the time features for this app, try different solutions using python packages, and finally suggest the “aviation method” for solving all the problems before they even happen.

Learn the ins and outs of Python typing implementation in this PyCon lecture. Discover why it's necessary and how to avoid common mistakes.

In this PyCon lecture, we'll explore Python typing implementation and why it's an important tool for any Python developer. We'll cover the basics of how to implement typing correctly, as well as some common mistakes to avoid. We'll also delve into some popular typing-related tools like mypy, dataclass, and pydantic. You'll come away with a solid understanding of Python typing and how to use these tools to improve your code quality and maintainability. Whether you're new to Python or an experienced developer, this lecture is a must-see for anyone looking to improve their coding skills.

Dependency management, virtual environments, publishing packages - all of these activities are a part of daily life when working with Python, but are too often high friction and fragile. Recent advancements make the situation much better.

The pyproject.toml file is the new standard for declaring Python projects, and it tells a story. Evidence for its significance is the tomllib built-in module introduced in Python 3.11. Many Python tools and linters adopted it as their primary configuration source. You should also use it - whether you publish a library, build an application or even write scripts/notebooks.

This talk will take us through the story of modern Python build systems and package managers. We'll learn where they come into play in the development life-cycle, their benefits, and their limitations. Specifically, we will mention: Hatch, Poetry, PDM, Flit, and Rye.

We will cover dependency resolution and locking with .lock files and how to utilize them for reproducible environments and CI environment caches, facilitating development across large teams.

We will migrate together a Python project from the legacy setup.py file with many additional files into a single pyproject.toml with Poetry.

https://jond01.github.io/blog/pycon-il-23/

The talk would start with explaining what spark is. what problems it solves, and why you might want to use it. Then I'll describe common anti patterns, especially with the data engineering/science related code. and what you should probably do instead

Pyspark, spark’s python interface is a potent data processing tool and potentially very high performing. This talk is about PYSpark's strong points and how common anti-patterns abuse and hurt PYSpark applications' performance, forcing you to throw more money and lose many of spark benefits. But there is a better way, using native pyspark tools and patterns that I’ll present

This talk will review some of the most common pitfalls that can cause otherwise perfectly good Pandas code to grind to be too slow for any time-sensitive applications, and walk through a set of tips and tricks to avoid them.

Writing performant pandas code is not an easy task, in this talk I will explain how to find the bottlenecks and how to write proper code with computational efficiency, and memory optimization in mind.

Maintaining world scale HD maps requires massive compute jobs. We use PySpark and python to map the world, making it performant and cost efficient - as you must at such scale. A glimpse on how we develop and optimize algorithmic code in Mobileye REM

REM group in Mobileye is tasked with the challenge of creating and maintaining a high definition map at world scale with cm level accuracy of all road geometry and semantic elements to enable fully autonomous driving.

The map is constructed from crowd sourced anonymized data of millions of driving assistance systems running computer vision processes in consumer vehicles.

In this talk we will share stories from the trenches on how we optimized python workloads to run distributed big data processing. We will discuss: - how we approach these kinds of issues - tools we use to identify and optimize algorithmic python code - examples of how to write algorithmic code that runs 10x times faster leveraging a range of tools and technologies

Dealing with database migrations for developers can be pretty tricky. Without a dedicated tool, this can be a frustrating task. Alembic is a tool to deal with this issue.

Dealing with database migrations for developers can be pretty tricky. Without a dedicated tool, this can be a frustrating task.

Python has frameworks that combine code and database management such as Django. What if we prefer using a simpler framework for running our code, like Fast Api that does not contain a database management functionality? Alembic tool is a great solution for that. Alembic tool helps us manage our database by creating and running migrations automatically.

This talk will go over the basics: - Start working with alembic on a Fast Api project. - The basic functionality that this tool provides and its' main features. - What are the pros and cons of working with this tool?

Both in personal and work lives- automation can ease our tasks. Together we will explore different times when to use it, review some code, and learn how to think automation and how to implement it. Let’s discover the amazing power of automation!

What is your first association when you hear “automation”? Is it QA automation? Business Process Automation? Data Analysis? If we explore it deeply, we can reveal many undiscovered layers- many uses of automation in many aspects of life.

Ever since my first position as an EDA Engineer - Electronics Design Automation Engineer – many years passed, I have been to many development positions, and I still find myself using these abilities I acquired, to improve my performance and excel in my tasks, both at work and my personal projects. Have you ever looked for a solution to ease your tasks in your personal or work life? Have you ever tried to think Automation? Can we really use it outside the defined world of automation jobs?

In this entry level lecture we will review together some real cases I dealt with and how I used Automation as a solution and improvement tool - automating repetitive commands, automating manual work, extracting information out of a big data and even writing a script that writes a reusable code. We will analyze the issues, review some code and discover the Power of Python in these tasks, and together we will train “automation thinking”.

Join me on the full fascinating tour of the automation mindset, and come out with a new and improved skill in your “bag of tricks”.

Unlock the potential of clean code in data science! Join our enlightening talk inspired by "Clean Code" by Robert C. Martin. Enhance readability, maintainability, and efficiency in your data science projects.

Python is a powerful and flexible language. However, its flexibility can sometimes lead to suboptimal code. For instance, consider this snippet taken from a real-world codebase: {v: [] for v in [a for b in p for a in b]}. Many data scientists who use Python on a daily basis lack traditional software engineering education, resulting in code that may be difficult to maintain and debug.

Fortunately, best practices of clean code in software engineering have existed for many years and can help to avoid these problems before they occur. In this talk, we will review fundamental concepts from the influential book "Clean Code" by Robert C. Martin. The book was written in Java, but I assure you that I have enough examples of bad code also in Python :) We will discuss when and how to incorporate these concepts into your daily work, providing practical examples of clean code dos and don'ts in Python.

If you're a team lead, software developer, or data scientist interested in producing better code and spending less time debugging, this talk is for you. Join me to learn how to level up your team's skills and write maintainable, efficient code that will save you valuable time.

Missing data is common when working with real-world data. In this talk, we will visualize missing data and discuss patterns of missing data and how to handle them.

Did the cat eat your CSV file? Did it eat only every third record? Missing data is prevalent in real-world data and can be missing for various reasons.

In this talk, we will talk about the different patterns of missing data and what are the best practices for handling each. In addition, we will show how to visualize missing data as part of our data exploration phase to understand our data better.

As python is the leading programing language for data scientists and data analysts, we will use pandas, missingno, scikit-learn and other tools to demonstrate those ideas and explore the data.

If you ever had a test that started to fail unprovoked or once in a while for no apparent reason, it's possible your code is relying on something that is non-deterministic. In this talk, I propose a simple way to address these challenges in Python!

There are many sources of nondeterminism in code such as randomness, IO, environment variables, databases and many more. Dependency injection is a pattern that provides a way to control nondeterminism in code and make tests easier to write.

In this talk I'll demonstrate how nondeterminism can pose challenges even in simple functions, and how to use dependency injection to address these problems!

This talk will provide best practices for building authorization with Python. To do so, he will show how to use existing tools in the ecosystem: OPA, OSO, OPAL, Zanzibar and others.

Authorization is a critical part of any application due to users' greater privacy awareness and new privacy standards like GDPR. Therefore, developers today are required to build complex authorization logic. However, most developers lack the expertise to build authorization with Python correctly, and they rebuild authorization from scratch repeatedly. This talk will provide best practices for building authorization with Python. To do so, he will show how to use existing tools in the ecosystem: OPA, OSO, OPAL, Zanzibar and others. Finally, he will provide recommendations about how to replace the pain of building authorization with implementing a simple and scalable solution.

In March, OpenAI discovered a vulnerability in redis-py, leading to potential data exposure. This is the inside story of the vulnerability: learn how it was solved, addressed, and fixed - in an open source community.

Ever experience a bug in a python library? What about a library that's downloaded nearly 30 million times a month? This talk focuses on a vulnerability - one that caused data leakage in some services.

Attend this talk this talk, to learn about the vulnerability that brought down the world's largest AI framework. Walk away with the tools to address these bugs in the future, learning how the library was debugged, and how the fix was tested. This talk will cover the technical detail behind how the fix was applied in the open, but also the process of working with security issues. Learn how to work with an Open Source community, while addressing a critical bug and keeping your cool.

Every line of code written is open-source, and in the public record, today.

Security is most valuable when it brings solutions which are specific to your use case, but enforcing them is hard. In this talk, we will show how to enforce internal python security guidelines that generic tools won’t find, in a repeatable way.

The best software security solutions to your security requirements and challenges are specific to your use case, self-service and don't impede development velocity. This will often include creating custom, safe versions of functionality or not allowing the use of certain library functions which we know are dangerous. However, making sure that this guidance is constantly followed can be problematic.

Whilst standard automated tools such as bandit or pylint may help discover generic vulnerabilities such as insecure use of pickle or security errors/misconfigurations such insecure autoescape configuration, how can we verify that our custom solutions and guidance have been correctly implemented and more importantly, stay that way, without writing complicated custom rules for these tools?

In this talk we will discuss examples of custom solutions like this but more importantly, we will demonstrate how to continuously verify that the code remain in a secure state with these solutions implemented, on an ongoing basis. We will demonstrate using simple rule syntax provided by the free, open-source, Semgrep tool.

Some example scenarios we’ll discuss:

• Verifying we are not using a disallowed function in a 3rd party library.
• Checking for the presence of security sensitive decorators in all places with allowed exceptions, etc.
• Looking for a function that is called in an unsafe way.

You should leave with ideas for how you can have custom and specific security guidelines that match your situation and how you can use a simple rule syntax to verify them as well as solve other similar code analysis problems.

Join us to unravel the mystery behind facial recognition technology and explore potential security vulnerabilities. Let us show you how we broke into a biometric database with a sophisticated, Python, machine-learning, brute-force attack!

Magic happens every time you take your phone out of your pocket. Somehow, just by looking at the screen, your phone recognizes you (and only you) and magically unlocks.

Have you ever stopped for a minute and thought to yourself - How does that even work? And maybe more importantly, how secure is it?

In this session, we're going to understand how facial recognition works under the hood. We'll dive into some potential security problems, and we'll show you how we were able to break into a biometric database built on the Dlib-python-library by applying a sophisticated brute-force attack. The results will surprise you.

Outline: - Intro - The magic of Face ID [1 min] - Under the hood - How Deep Learning Face ID works [4 min] - Comparing faces - Exploring the Face-Space [3 min] - Recap - Tying it all together [2 min] - Exploitation - Weird behavior & Possible weakness? [4 min] - Shocking results! - [1 min] - The Attack - Our attack on a Biometrical database [4 min] - Summary [1 min]

New post quantum python libraries are here to replace the old ones. Simply replacing old with new may prove wrong, but there is a way to do it right.

The post-quantum era is upon us, and new cryptographic algorithms are now available. Python developers that use encryption, signatures, or other cryptographic operations in their code must update their Python code. There are already Python implementations, but they can be problematic to use as a drop-in. In this session, We will learn about post-quantum cryptography algorithms and why we should use them. We will also learn to combine the old and new algorithms securely, ensuring we maintain our classical assurance and certification while preparing for the quantum challenges.