https://www.youtube.com/watch?v=az-va7ofef8&embedable=true Introduction For those not familiar with Ambient TVs, it’s a way to soften the jump from the edge of the TV screen and its immediate surroundings to provide a more immersive experience. I had some LED lights lying around and decided to see if it was possible to control the lights through code and, in turn, make my computer screen an ambient monitor. Whilst I wanted to use it for my monitor, it can be used anywhere and with whatever colors you can send it, including other features your lights may have, such as audio reactions or random patterns. I’ve been meaning to write this post for a while as I’ve been using it on an earlier monitor, but I never got around to adding it to my new monitor, so I documented it as I went along for anyone who might find it useful. So let’s get to it! (Please note, LED lights are likely to be Bluetooth Low Energy (BLE), so your computer will need to support BLE in order to interact with them). The full code is on . GitHub High Level Steps Find out what commands the LED light’s Bluetooth receiver accepts Send commands to the LED lights via my computer’s Bluetooth Obtain the dominant color of the current screen Send the dominant color to the LED lights Prerequisites Bluetooth-supported RGB LED lights and accompanying app (I’m using Android, iOS would likely require an alternative approach than the one described here, but it should be possible to use Wireshark directly to monitor Bluetooth traffic). I've attached these lights to the back of my monitor Wireshark (specifically adb.exe) Android’s SDK tools Developer tools (I’ll be using Python 3.10, though any 3.x versions should work, but the principles should be the same whatever for language you prefer) A device to send BLE commands from (e.g. a laptop that supports BLE) Getting Bluetooth data The first step we need to do is to ensure that the app that comes with the lights is working as expected. This can easily be tested by running the light’s original app and making sure that the lights react accordingly depending on the on/off/lighting buttons you’re pressing on your app. We do this because we will shortly be pressing and detecting the specific codes sent to the Bluetooth receiver on the lights. There are two approaches that I could take. One was to decompile the app’s JAR file and find the codes that were being sent, but I wanted to learn more about the Bluetooth protocol, so I opted to log all Bluetooth activity on my Android and extract it from there. Here’s how: Enable on your Android device. Developer Options Enable Bluetooth HCI snoop log (HCI stands for Host-Controller Interface). You can find this option in or search for it in settings as in the image below. Settings > System > Developer We now need to perform specific actions so we can identify what each action sends to the light’s Bluetooth receiver. I’m going to keep it simple to On/Red/Green/Blue/Off, in that order, but if your lights support other features, you can toy around with those too. Run the app and press On, Red, Green, Blue, and Off. It may also be useful to keep an eye on the approximate time to make it easier to filter if you have a lot of Bluetooth activity on your device. Turn Bluetooth off so we don’t get any more noise. In the following steps, we’ll analyze the Bluetooth commands and, as we know the order of what we pressed, we can find out which values correspond to which button press. We now need to access the Bluetooth logs on the phone. There are several ways to do this, but I will generate and export a bug report. To do this, enable USB Debugging in the phone’s Settings, connect the phone to the computer, and use the adb.exe command line tool. adb bugreport led_bluetooth_report This will generate a zip file on your computer’s local directory with the filename “ ”. You can specify a path if you prefer (e.g. C:\MyPath\led_bluetooth_report”) led_bluetooth_report.zip Within this zip are the logs that we need. This may vary from device to device (please comment if you found it elsewhere on your device). On my Google Pixel phone, it was in FS\data\misc\bluetooth\logs\btsnoop_hci.log Now we have the log files, let’s analyze them! To do this, I decided to use Wireshark so start Wireshark and go to File...Open... and select the btsnoop_hci log file. Whilst it may look daunting, let’s make it easy for ourselves to find what we’re looking for by filtering the on which is the Attribute Protocol in the . The attribute protocol defines the way two BLE devices talk to each other, so this is what we need to help find how the app talks to the lights. You can filter the logs in Wireshark by typing in the “ ” bar near the top and press Enter BTL2CAP 0x0004, Wireshark source code btl2cap.cid == 0x0004 Apply a display filter Now we have filtered the log; it should make looking for the commands easier. We can look at the timestamps (Go to View…Time Display Format…Time of Day to convert the time if it’s the wrong format). We want to look at the Sent Write Command logs as those are the ones where we sent a value to the lights. Assuming your most recent time is at the bottom, scroll down to the last five events. These should be On, Red, Green, Blue, and Off in that order, with Off being last. Take note of the Destination BD_ADDR, as we'll need that shortly, and put on your Sherlock Holmes hat, as this is where we need to unlock the pattern of how the colors and on/off commands are encoded within the message. This will vary depending on the light manufacturer but here’s the list of values I got for my device: On: 7e0404f00001ff00ef Red: 7e070503ff000010ef Green: 7e07050300ff0010ef Blue: 7e0705030000ff10ef Off: 7e0404000000ff00ef These are clearly hexadecimal values and if you look carefully, you’ll see there are some fixed patterns. Let’s split the patterns out as this should make things much clearer. On: 7e0404 ff00ef f00001 Red: 7e070503 10ef ff0000 Green: 7e070503 10ef 00ff00 Blue: 7e070503 10ef 0000ff Off: 7e0404 ff00ef 000000 For those familiar with hexadecimal values of pure red, green, and blue, you’ll know that the values are #FF000, #00FF00, and #0000FF, respectively, which is exactly what we can see above. This means we now know the format to change the colors to whatever we want! (or at least to what the lights themselves are capable of). We can also see that On and Off have a different format from the colors and are similar to each other, with On having f00001 and Off having 00000. That’s it! We now have enough information to start coding and interacting with the lights. Connecting to LED lights There are three key things we need: The address of the device (this is the Destination BD_ADDR from above) The values to send to the device (the hexadecimal values obtained above) The characteristic we want to change. A Bluetooth LE characteristic is a data structure that essentially defines data that can be sent between a host and client Bluetooth devices. We need to find the characteristic (a 16-bit or 128-bit UUID) that refers to the lights. There are some commonly used assigned numbers that can be found here but unless the device conforms to those, they could be using a custom UUID. As my lights aren’t in the assigned numbers list, let’s find it via code. I’m using Python 3.10 and . Ensure Bluetooth on your computer is turned on (no need to pair with the device, we’ll connect to it through code). Bleak 0.20.1 # Function to create a BleakClient and connect it to the address of the light's Bluetooth reciever
async def init_client(address: str) -> BleakClient:
    client =  BleakClient(address)  
    print("Connecting")
    await client.connect()
    print(f"Connected to {address}")
    return client

# Function we can call to make sure we disconnect properly otherwise there could be caching and other issues if you disconnect and reconnect quickly
async def disconnect_client(client: Optional[BleakClient] = None) -> None:
    if client is not None :
        print("Disconnecting")
        if characteristic_uuid is not None:
            print(f"charUUID: {characteristic_uuid}")
            await toggle_off(client, characteristic_uuid)
        await client.disconnect()
        print("Client Disconnected")
    print("Exited")

# Get the characteristic UUID of the lights. You don't need to run this every time
async def get_characteristics(client: BleakClient) -> None:
    # Get all the services the device (lights in this case) 
    services = await client.get_services() 
    # Iterate the services. Each service will have characteristics
    for service in services: 
        # Iterate and subsequently print the characteristic UUID
        for characteristic in service.characteristics: 
            print(f"Characteristic: {characteristic.uuid}") 
    print("Please test these characteristics to identify the correct one")
    await disconnect_client(client) I’ve commented on the code, so it should be self-explanatory but essentially, we connect to the lights and find all the characteristics it exposes. My output was: Characteristic: 00002a00-0000-1000-8000-00805f9b34fb
Characteristic: 00002a01-0000-1000-8000-00805f9b34fb
Characteristic: 0000fff3-0000-1000-8000-00805f9b34fb
Characteristic: 0000fff4-0000-1000-8000-00805f9b34fb A quick Google of the first two UUIDs shows this refers to the name and appearance of the service, which is irrelevant for us. However, the third and fourth seem the most suitable with the third ( ) being the write characteristic according to . Excellent, we now have the characteristic we need for this particular device to write to with a value (the color hexadecimal). 0000fff3-0000-1000-8000-00805f9b34fb this page Controlling LED lights We finally have all the pieces this we need. At this stage, you can get creative with what color input you’d like to use. You could, for example, connect the lights to a trading market API to change colors according to how your portfolio is doing. In this case, we want to make our monitors ambient aware, so we need to obtain the dominant color of the screen and send that through. There are many ways to do this, so feel free to experiment with whatever algorithms you would like. One of the simplest approaches would be to iterate every X number of pixels across the screen and take an average while more complicated solutions would look for colors human eyes perceive to be more dominant. Feel free to comment on any findings you’d like to share! For the sake of this blog post, I’m going to keep it simple by using the library’s get_dominant_color method. fast_colorthief '''
Instead of taking the whole screensize into account, I'm going to take a 640x480 resolution from the middle. 
This should make it faster but you can toy around depending on what works for you. You may, for example, want
to take the outer edge colours instead so it the ambience blends to the outer edges and not the main screen colour 
'''
screen_width, screen_height = ImageGrab.grab().size #get the overall resolution size 
region_width = 640
region_height = 480
region_left = (screen_width - region_width) // 2
region_top = (screen_height - region_height) // 2
screen_region = (region_left, region_top, region_left + region_width, region_top + region_height)

screenshot_memory = io.BytesIO(b"")

# Method to get the dominant colour on screen. You can change this method to return whatever colour you like
def get_dominant_colour() -> str:
    # Take a screenshot of the region specified earlier
    screenshot = ImageGrab.grab(screen_region)
    '''
    The fast_colorthief library doesn't work directly with PIL images but we can use an in memory buffer (BytesIO) to store the picture
    This saves us writing then reading from the disk which is costly
    '''
    
    # Save screenshot region to in-memory bytes buffer (instead of to disk)
    # Seeking and truncating fo performance rather than using "with" and creating/closing BytesIO object
    screenshot_memory.seek(0)
    screenshot_memory.truncate(0)
    screenshot.save(screenshot_memory, "PNG") 
    # Get the dominant colour
    dominant_color = fast_colorthief.get_dominant_color(screenshot_memory, quality=1) 
    # Return the colour in the form of hex (without the # prefix as our Bluetooth device doesn't use it)
    return '{:02x}{:02x}{:02x}'.format(*dominant_color) The code is commented, so hopefully, it should be clear as to what’s happening, but we’re taking a smaller region of the screen from the middle and then getting the dominant color from that region. The reason I’m taking a smaller region is for performance; fewer pixels would need to be analyzed. We’re almost there! We now know what to send it and where to send it. Let’s finish the last major part of this challenge which is to actually send it. Fortunately, with the Bleak library, this is quite straightforward. async def send_colour_to_device(client: BleakClient, uuid: str, value: str) -> None:
    #write to the characteristic we found, in the format that was obtained from the Bluetooth logs
    await client.write_gatt_char(uuid, bytes.fromhex(f"7e070503{value}10ef"))

async def toggle_on(client: BleakClient, uuid: str) -> None:
    await client.write_gatt_char(uuid, bytes.fromhex(ON_HEX))
    print("Turned on")

async def toggle_off(client: BleakClient, uuid: str) -> None:
    await client.write_gatt_char(uuid, bytes.fromhex(OFF_HEX))
    print("Turned off") As we discovered from the logs, each color has a fixed template, so we can use f-strings to hardcode the common part and simply pass a hexadecimal of a color for the value in the middle. This can be called from our loop. On and Off had unique hexademicals so I created individual functions and passed in a constant value that contained the relevant hex. while True: 
    # send the dominant colour to the device
    await send_colour_to_device(client, characteristic_uuid, get_dominant_colour())
    # allow a small amount of time before update
    time.sleep(0.1) And there we have it; our Bluetooth LED lights are now controlled by the colors on the screen, creating our own Ambient Monitor. You can see the full code on , which has a small amount of infrastructure code that wasn't specific to this post. I’ve tried to comment the code to be self-explanatory but feel free to ask any questions or make suggestions. GitHub Hopefully, this gives you an idea on how you can start getting creative with your LED lights. Future Improvements - My light strips can only have one color at a time, but I have another set that can have four quadrants, each with its own colors. This means it could be possible to have sections of the monitor match four sections on screen giving an even more accurate ambiance setting. Those lights run on Wifi instead of Bluetooth and could be a future project. Multi-colour – to keep it simple, I just looked for the color changing and the on-and-off commands. However, this can easily be improved by detecting the brightness control commands and throwing that into the color algorithm. Brightness - As we want to get the lights to change in realtime, performance is critical. There are some complex algorithms to detect which color would be considered the most dominant, especially when perceived by humans (which leads to a whole world of color conversions). However, since this needs to run quite quickly, there needs to be a balance between performance and accuracy. A future improvement could be to try and access the graphics card directly to read from the buffer rather than analyzing the pixels on the screen directly. If this is possible, you would also eliminate the time taken from the graphics buffer to the screen which could optimize the reaction of the lights. Performance Feel free to comment below if you have any feedback or questions. Also Published Here

This story contains new, firsthand information uncovered by the writer.

Reverse Engineer LED Lights with Python to Convert your Computer Monitor to an Ambient Monitor

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Are your PDFs Actually Redacted? Double Check!

$PEPE, a Purple Lamborghini, and More: The Story Continues

102 Most Important Webism Quotes by Unknown Authors

100 Days of AI Day 3: Leveraging AI for Prompt Engineering and Inference

100 Days of AI, Day 8: Experimenting With Microsoft's Semantic Kernel Using GPT-4

10 Tips to Take Your ChatGPT Prompts to the Next Level

Are your PDFs Actually Redacted? Double Check!

$PEPE, a Purple Lamborghini, and More: The Story Continues

102 Most Important Webism Quotes by Unknown Authors

100 Days of AI Day 3: Leveraging AI for Prompt Engineering and Inference

100 Days of AI, Day 8: Experimenting With Microsoft's Semantic Kernel Using GPT-4

10 Tips to Take Your ChatGPT Prompts to the Next Level

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps