Converting bytes to strings in Python is a common task when working with binary data. We will learn some of the best methods to do it with examples and code. Before that, let's revise what is sequential data.
Sequential Data refers to any data that contain elements that are ordered into sequences. Programmers use it extensively for many use cases and there are numerous ways to store sequential data in python. The six sequential data types in python are:
- Strings
- Byte Sequences (bytes objects)
- Byte Arrays (bytearray objects)
- Lists
- Tuples
- Range Objects
What is the bytes() function?
Simply, a byte is a unit of data that typically consists of 8 bits and can represent a single character or number. In Python, bytes is a built-in data type that represents a sequence of immutable, ordered, and indexable 8-bit integers (0 to 255).
Bytes are usually created using the bytes() function or the b prefix with a normal string syntax. The bytes() function in Python is a built-in constructor that creates a byte object from an iterable of integers, a string, or an object that implements the buffer protocol. It returns a byte object which is an immutable sequence of integers in the inclusive range of 0 to 256.
Here is an Example:
x = bytes('Example Byte', 'utf8') print(x)
Output:
b'Example Byte'
Another way to create a byte is via the b prefix to a normal string. Writing a b before a string will create a byte instead of a string. Here's the syntax:
x = bytes('Example Byte', 'utf8') print(x)
Output:
b'Example Byte'
To verify, you can call the type() function to see the data type of the variable in both cases.
type(x)
Output:
<class 'bytes'>
The above output shows that a byte has been successfully created.
How to Convert Bytes to String?
In programming, there could be plenty of scenarios where you might need to convert a byte to a string. For example, Due to bytes’ inherent nature of being immutable, converting them to strings should help if one needs to change the value.
Python provides various approaches to convert bytes to strings. Let us explore each approach one by one:
1) decode() function
This function decodes different types of encoding of strings to a normal string. The decode() function in python can be used to take our data encoded in bytes format and decode it to convert the data to string format.
Here’s the code for the implementation:
x = b'Example' print(x) print(type(x)) x = x.decode() print(x) print(type(x))
Output:
b'Example' <class 'bytes'> Example <class 'str'>
Here, we created a byte b’Example’ and assigned it to a variable x. In line 2 and 3, we print the value of x and the type of x respectively. The output we get is the byte value and the type of x which is in bytes. Then we use the decode() function to decode x and now when we print x, we get “Example”, a string and when we check the type using the type() function, we can see that the type is now in string.
Similarly, we can decode byte arrays with the same function and convert them into strings. Byte arrays are mutable sequences of bytes. Here’s how the decode function can be applied on bytearrays:
x = bytearray([1,2,3,4,5]) print(x) x = x.decode() print(x)
Output:
bytearray(b'\x01\x02\x03\x04\x05') '\x01\x02\x03\x04\x05'
Here, the decode function is being used to convert a byte array of a list [1,2,3,4,5] into a string.
2) str() function
The built-in str() function converts any compatible data into a string. We have to pass another argument ‘UTF-8’ to convey to the function that the string is formatted in UTF-8 format so that it can convert to the said format.
x = b'Example' print(x) x = x.str(x, 'UTF-8') print(x)
Output:
b'Example' Example
3) Using codecs.decode()
The Codecs module contains the base classes for the standard encoders and decoders. Codecs support numerous datatypes and data conversions and one of them is bytes. Codecs library provides a decode() function that can decode any datatype into a string. The decode() function of codecs can be accessed via ‘codecs.decode()’ after importing the module. Here’s the code:
import codecs x = b'Example' print(x) x = codecs.decode(x) print(x)
Output:
b'Example' Example
The codecs.decode() function takes in arguments of compatible datatypes and decodes them into strings. Bytes can be converted to strings in this way. Similar to codecs, many libraries offer a decoding method. A popular library called Pandas, which is used for working with datasets, offers such a decode method to decode bytes to strings.
4) str.decode() function in pandas
Pandas is a library extensively used for data analytics for tasks such as cleaning, exploring, and manipulating data. The task of converting bytes to strings comes under data manipulation due to the conversion aspect and pandas provides a function for it. Here is the code for the data manipulation in pandas:
import pandas as pd dic = {'column' : [b'ExampleOne', b'ExampleTwo', b'ExampleThree']} df = pd.DataFrame(dic) print(data['column']) x = data['column'].str.decode("utf-8") print(x)
Output:
0 b'ExampleOne' 1 b'ExampleTwo' 2 b'ExampleThree' Name: column, dtype:object 0 ExampleOne 1 ExampleTwo 2 ExampleTwo Name: column, dtype:object
Here, we accessed the column of bytes using dataframe indexing offered by Pandas and we used the .str.decode() function with a “utf-8” parameter to convert every value of the dataframe to a string.
Conclusion
Encoding and decoding are essential knowledge one must possess to be a good python programmer because of how often encoding and decoding could be necessary. From Data Cleaning to Data Encryption to much simpler use cases, encoding and decoding are highly used in python.
Now that we have gone through different sequential datatypes, an in-depth understanding of bytes, and the conversion of bytes to strings in python via various methods and libraries, we built a solid understanding of the encoding and decoding of bytes.
