Pandas shift()

import pandas as pd

# create a sample DataFrame
data = {'A': [1, 2, 3],
        'B': [4, 5, 6]}

df = pd.DataFrame(data)

# use shift() to move values of column A by one period down
shifted_a = df['A'].shift(periods=1)

print(shifted_a)

'''
Output

0    NaN
1    1.0
2    2.0
Name: A, dtype: float64
'''

shift() Syntax

The syntax of the shift() method in Pandas is:

obj.shift(periods=1, freq=None, axis=0)

shift() Arguments

The shift() method takes the following arguments:

• periods (optional) - number of periods to move, can be positive or negative
• freq (optional) - frequency on which to shift the data; only relevant for objects that have a frequency set, such as DatetimeIndex
• axis (optional) - axis along which to shift (0 or 1)

shift() Return Value

The shift() method returns an object (Series or DataFrame) with the data shifted by the specified number of periods.

Example 1: Shift Entire DataFrame Down

import pandas as pd

# create a DataFrame
data = {
    'Values': [10, 20, 30, 40]
}

df = pd.DataFrame(data)

# shift entire DataFrame down by one period
shifted_df = df.shift(periods=1)

print(shifted_df)

Output

   Values
0     NaN
1    10.0
2    20.0
3    30.0

In the example above, we shifted the entire DataFrame down by one period, introducing a NaN value at the first position.

Example 2: Shift DataFrame Up

import pandas as pd

# sample DataFrame
data = {'Values': [10, 20, 30, 40]}
df = pd.DataFrame(data)

# shift DataFrame up by one period
shifted_df = df.shift(periods=-1)

print(shifted_df)

Output

   Values
0    20.0
1    30.0
2    40.0
3     NaN

Here, we shifted the DataFrame up, so the first value is dropped, and a NaN value is introduced at the end.

Example 3: Shift Data Horizontally

import pandas as pd

# sample DataFrame
data = {'A': [1, 2, 3],
        'B': [4, 5, 6],
        'C': [7, 8, 9]}

df = pd.DataFrame(data)

# shift data horizontally by one period
shifted_df = df.shift(periods=1, axis=1)

print(shifted_df)

Output

    A  B  C
0 NaN  1  4
1 NaN  2  5
2 NaN  3  6

In this example, we shifted the data in the DataFrame to the right by one column, leading to the introduction of NaN values in the first column.

Example 4: Shift Frequency

import pandas as pd

# sample DataFrame with datetime index
date_rng = pd.date_range(start='2020-01-01', end='2020-01-03', freq='D')
data = {'Value': [10, 20, 30]}

df = pd.DataFrame(data, index=date_rng)

# shift data by one day using freq argument
shifted_df = df.shift(freq='D')

print(shifted_df)

Output

            Value
2020-01-02     10
2020-01-03     20
2020-01-04     30

In this example, we shifted the entire DataFrame by one day using the freq='D' argument in the shift() method.

The shift() method is beneficial when analyzing time-series data, for creating lag or lead variables, or when repositioning data in the DataFrame.