How Microsoft used ChatGPT to fly drones and robots

Microsoft believes that ChatGPT can revolutionize the robotics industry. Thanks to the artificial intelligence of OpenAI, it is indeed possible to control drones and robots without being a programming expert.

Keen to take the lead in the AI ​​race, Microsoft pumped a fortune into OpenAI, the start-up behind ChatGPT. Backed by several massive investments, the Redmond giant quickly integrated the chatbot into Bing, its search engine, and Edge, its web browser. 


Microsoft intends to continue its momentum by incorporating ChatGPT into other flagship products, such as Outlook, Word, or Powerpoint.

The American firm does not wish to confine ChatGPT to the software sector. Microsoft also aims to use the advances of OpenAI to revolutionize the world of robotics. In a report published a few days ago, the company reveals that it used OpenAI’s chatbot to control “robot arms, drones, and home assistant robots”.

In detail, Microsoft relied on ChatGPT to control a flying drone. At the request of the AI, the craft performed a zigzag in the air before taking a selfie using its integrated camera. By following ChatGPT’s orders, the drone also managed to reach a specific location while avoiding obstacles in its path.

The engineers also used the chatbot to do tests in Microsoft AirSim, the simulator for drones. In its labs, Microsoft has also used ChatGPT to order a robot arm to stack blocks. The machine succeeded in stacking these blocks following a very specific pattern.



Facilitate communication between man and drones (machine)


Microsoft explains that it wants to facilitate the interaction between human beings and robots by relying on the OpenAI language models. To achieve this, ChatGPT must imperatively “think beyond the text and reason about the physical world”. 

Clearly, the AI ​​must not be content to generate answers based on the requests of its interlocutor. She must understand what the requests entail from an environmental and physical law point of view.


Use ChatGPT as a translator


As part of its research, the American giant has developed “design principles”. Thanks to these different principles, Microsoft used ChatGPT to generate code capable of controlling a robot. A complete methodology has been developed by the group’s engineers based on these precepts. This details how it is possible to use a language model to facilitate the programming of a robot.

First of all, it is imperative to define a directory of the functions of the robot. Each function should have a descriptive title so that ChatGPT understands what it is. In the second step, the user writes a request in the interface of the chatbot describing the task that awaits the robot. As always, you have to give as much information as possible to the AI ​​to get the desired result.


When writing the request, the user must describe the objective of the task, indicate which functions described previously must be used, and specify any constraints. Based on the provided data, ChatGPT will translate the request into a programming language. 

The user can then evaluate the code provided and carry out a simulation as a precaution, before deploying it on the PLC. At the end of this process, the robot will have obeyed a command generated by ChatGPT at the request of its interlocutor. The user does not necessarily need to know a complex programming language to communicate with a robot or a drone.

Note that Microsoft isn’t the only tech titan looking into the usefulness of language models in the field of robotics. Google is currently working on similar experiments based on PaLM (Pathways Language Model), a language model of its own.

Source: Microsoft

Leave a Reply

Your email address will not be published. Required fields are marked *

Jobs, Loans, Scholarships and Admission Guide Portal