In Pandas, how do I remove DataFrame rows that contain None
or NaN
across all columns? How can I do this when these values are present in only some columns?
We can achieve both of these results using the DataFrame.dropna
method. For example:
import pandas from numpy import nan df = pandas.DataFrame( { "Test 1": [90, 10, nan, nan], "Test 2": [41, nan, 32, nan], "Test 3": [89, 35, 72, nan], "Test 4": [52, nan, nan, nan], } ) print(df) # output: # Test 1 Test 2 Test 3 Test 4 # 0 90.0 41.0 89.0 52.0 # 1 10.0 NaN 35.0 NaN # 2 NaN 32.0 72.0 NaN # 3 NaN NaN NaN NaN df_no_empty_rows = df.dropna(how="all") # drop rows containing all NaNs print(df_no_empty_rows) # output: # Test 1 Test 2 Test 3 Test 4 # 0 90.0 41.0 89.0 52.0 # 1 10.0 NaN 35.0 NaN # 2 NaN 32.0 72.0 NaN df_no_empty_values = df.dropna(how="any") # drop rows containing any NaNs print(df_no_empty_values) # output: # Test 1 Test 2 Test 3 Test 4 # 0 90.0 41.0 89.0 52.0
Get actionable, code-level insights to resolve Python performance bottlenecks and errors.
Create a free Sentry account
Create a Python project and note your DSN
Grab the Sentry Python SDK
pip install --upgrade sentry-sdk
import sentry_sdk sentry_sdk.init( "https://<key>@sentry.io/<project>", # Set traces_sample_rate to 1.0 to capture 100% # of transactions for performance monitoring. # We recommend adjusting this value in production. traces_sample_rate=1.0, )
Loved by over 4 million developers and more than 90,000 organizations worldwide, Sentry provides code-level observability to many of the world’s best-known companies like Disney, Peloton, Cloudflare, Eventbrite, Slack, Supercell, and Rockstar Games. Each month we process billions of exceptions from the most popular products on the internet.