Our objective was to control the robot remotely and not with a controller attached that is why we bought the ESP32 board.

• AsyncTCP by dvarrel
• ESP Async WebServer by Me-No-Dev
• ESP32Servo by Kevin Harrington, John K. Bennet
• EspSoftwareSerial by Dirk Kaar, Peter Lerup
• OttoDIYLib by Otto DIY, Camilo Parra Palacio

With that in mind we arrive at our first decision: use Bluetooth or Wifi. If we use Bluetooth we can attach a Bluetooth controller and call it a day, but that way it is really difficult to work with the camera. So in reality we only had one choice from the start, to use WiFi.

Now going with the WiFi router we have to decide again between 2 possible options: connect to an existing WiFi or create an access point in the ESP32. Both options are valid, but the first one comes with a big drawback for us, and that is the necessity to flash the ESP32 whenever we switch places and also that we cannot connect to the university wifi without a lot of hassle.

With all of this sorted out we can continue to program knowing that we are gonna use WiFi in AP mode.

We just set up the AP in the ESP32 as follows:

const char* ssid = "WebWay";
const char* password = "123456789";

// Here we set the Ip
IPAddress Ip(192, 168, 1, 1);
IPAddress NMask(255, 255, 255, 0);

// Create AsyncWebServer object on port 80
AsyncWebServer server(80);

And start it with the next lines:

  // Connect to Wi-Fi
  WiFi.mode(WIFI_AP);
  WiFi.softAP(ssid, password);
  delay(100);
  WiFi.softAPConfig(Ip, Ip, NMask);

  // Print ESP32 Local IP Address
  Serial.print("IP Address: http://");
  Serial.println(WiFi.softAPIP());

Now we know how to connect to the ESP32 from a computer or phone without using a cable, but nothing happens when we do it.

In order to display something we will need to use HTML and CSS to create the basic frontend and then JavaScript to send data to the ESP32 and to increase that basic functionality.

Thankfully there are multiple examples of how to control a motor from a Webpage

or how to display the camera image

After trying multiple examples we reached the conclussion that none of them were what we were looking for, ones because they were not expandable at all and others because of the sheer amount of lines of code required (more than 3.000). So we just took some ideas from here and there, like the ability to rotate the image in the Webpage and not in the ESP32.

Looking for something simple and asynchronous we found that Asynchronous Websockets was what we wanted, because they can be implemented in just a couple of lines of code and is "infinitely" expandable and also can be connected to the frontend just as simple.

Now with the ability to send data from the ESP32 to the Webpage we can start to try to display an image.

As the title says this was not fun. Using the example code worked perfectly but it was really complicated and long, and as we did not need to focus a lot on the software part we decided to not use that approach, because with it we would have been limited to only show the camera or spend weeks trying to paint stuff on top.

Then the next approach was to use image, this may not be the best approach but it works as demonstrated in Unibotics.

Now the pipeline was as follows:

Frontend request photo -> Backend websocket recieves (Take photo) -> Takes photo and convertd it to base64 -> Sends the photo to the Frontend -> Frontend decodes it and sets it as the background -> Repeat every 100 ms

WE COULD ADD A DIAGRAM HERE

One issue that you may have thought off is that "can the ESP32 send HD image at 10 FPS?". And the answer is no, not even close. Using the example code it is capable of doing it but with the images approach it cannot.

The solution, reduce the quality of the image to 352 x 288. This solves most of the problems, but if the WiFi connection fails a bit the frontend image can flicker because of images being corrupt. This can be seen in the videos below.

I will not enter into much detail in this section because most of the stuff is basic html and CSS, but to do a recap here are the most important parts:

• The camera image is the background and a JS script updates it every 0.1 seconds
• On top of the image there is custom made svg that tries to simulate a camera visor. This svg can change from white to black with a button.

• A theme switcher button. Changes all black components to white and the opposite. It is also custom made.

• A rotate image button that rotates the image 180 degrees. It is also custom made.

• A clock. It tells time.
• Action buttons that control what the robot does. They comunícate with the ESP32.

The Webpage may not render perfectly because of your screen size or even browser, id you want to fix this then change the CSS values to your own liking.

With everything else done we reach the end of our destination and the only thing missing is to move the robot.

First we need to define the actions that the robot will be capable of doing and in our cade we went with the basics:

1. STOP: Stops the robot
2. FWD: goes forward
3. BWD: goes backwards
4. ROT CLK: rotates clockwise
5. ROT ICLK: rotates counterclockwise

With this we create the actions buttons in the frontend and in the backend we tell the websocket to listen to that actions

• Structure of a button in html

    <div
      id="bt5"
      class="button"
      onclick="document.getElementById('bt5').classList.toggle('selected');
      document.getElementById('bt1').classList.remove('selected');
      document.getElementById('bt2').classList.remove('selected');
      document.getElementById('bt3').classList.remove('selected');
      document.getElementById('bt4').classList.remove('selected');
      var xhr = new XMLHttpRequest();xhr.open('POST', '/my_action', true);xhr.send();"
    >
      <div class="text">
        MY_ACTION
      </div>
    </div>

• Backend code to receive that action

void my_action(AsyncWebServerRequest *request) {
  Serial.println("My Action");
  do_action = MY_ACTION;
}

server.on("/my_action", HTTP_POST, my_action);

The last part will be to execute the requested action in a loop until the user changes it and what better way to do it than using a switch case in our main loop.

The first approach at moving it was using the Otto library as this robot is similar in shape to that one, and after calibrating the motors manually it "worked". It was not the best movement but at the very least it was finally moving. The code can be found ADD LINK HERE.

Uploading videosTesting6.mp4…

But we wanted to move a little bit more fluidly so our next objective was to make the robot move as the one it is based on, the Otto Ninja.

A big problem was that the Otto Ninja does not use any piece that we use, the motors are different, the controller is different and even the batteries. We tried using the movements defined in an example inside the Otto Ninja repository but after adapting all the code our calibration values were way off and the robot did not move at all because of the lack of weight and the ever changing weight distribution. Also it did not help that one motor decided to work randomly.

After printing the final pieces, putting everything in place and changing that pesky motor we calibrated everything properly until it worked. In the end the movement done by hand did not work because of the size of the head, so we used the one provided by the Otto library