17 lines
1.5 MiB
JSON
17 lines
1.5 MiB
JSON
|
|
[
|
||
|
|
{
|
||
|
|
"role": "user",
|
||
|
|
"content": "You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task. \n\n## Output Format\n```\nThought: ...\nAction: ...\n```\n\n## Action Space\n\nclick(start_box='<|box_start|>(x1, y1)<|box_end|>')\nleft_double(start_box='<|box_start|>(x1, y1)<|box_end|>')\nright_single(start_box='<|box_start|>(x1, y1)<|box_end|>')\ndrag(start_box='<|box_start|>(x1, y1)<|box_end|>', end_box='<|box_start|>(x3, y3)<|box_end|>')\nhotkey(key='')\ntype(content='') #If you want to submit your input, use \"\\n\" at the end of `content`.\nscroll(start_box='<|box_start|>(x1, y1)<|box_end|>', direction='down or up or right or left')\nwait() #Sleep for 5s and take a screenshot to check for any changes.\nfinished(content='xxx') # Use escape characters \\', \\\", and \\n in content part to ensure we can parse the content in normal python string format.\n\n\n## Note\n- Use Chinese in `Thought` part.\n- Write a small plan and finally summarize your next action (with its target element) in one sentence in `Thought` part.\n\n## User Instruction\nCould you help me set the image to Palette-Based?"
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"role": "user",
|
||
|
|
"content": [
|
||
|
|
{
|
||
|
|
"type": "image_url",
|
||
|
|
"image_url": {
|
||
|
|
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB4AAAAQ4CAIAAABnsVYUAAEAAElEQVR4nOzdd3wUxRcA8Dezu1fTQ3oCofcuvUixUBSVZkERFcSu2MFesf3EAopYqYpiQ7HQe+81tAAJgRTSL1d3Z35/7OW4XMul0Xxff798kr3dndnd2ePu3bs3JDo6GhBCCCGEEEIIIYQQQgih2kYvdgcQQgghhBBCCCGEEEIIXZkwAI0QQgghhBBCCCGEEEKoTmAAGiGEEEIIIYQQQgghhFCdwAA0QgghhBBCCCGEEEIIoTqBAehLSExMzMXuAkIIIYQQQrWmdevWX3755YMPPqj++eCDD86aNatVq1YXt1cIIYQQQuhCwgD0BRIVFRV4hYSEBFEUL0xnEEIIIYQQqmtDhgz55ptvVqxYMXv2bHXJ7NmzV65c+e233w4aNOji9g0hhBBC6Ipx77331vo+T5w4UYt7wwD0BaLRaMLDw/09mpiY2LJlS0rxclyKBj43493RKcLF7gZCCCGE0GUkLi7ujTfeuPvuu1esWPHuu+/u3r179+7d77zzzooVK8aNG/fWW2/FxcVd7D4ihBBCCF32Bg4cOGXKFPclOp2uGvshhDzyyCMREREeyyMiIp544glBqFFg7IJGPJM6D+zTzEBcf0tdH/tr5+opHf1uUHGFKm9+KdFqtaNHj54wYcKECRPuv//+t95668knn5wwYcL48eNHjBhBKU1JSSGEVL4jVFU1HidtBl7bJVkX+Npc1oMTIYQQQoHxeqM+3XU8bd6olNp77axrMW76hkMLbz+foMBj7v427UR6ueNLH+8Y4NtxQmLPJ75asuXgof0blnxyX48oDgAA2oT+j77/45odB9MObV+xcOptHcN8HY7Phho8vPBwekUHv76h+u80xowZs2zZsuPHj8+ZMyctLa137969e/c+cuTI7Nmzjx07tnz58ttvv73aO0cIIYQQQgDQrl27Tz/91D06TAj58ccf+/XrV9VdTZky5dprr/X50IABAzxi3FVV0xfRVXo5fu/0z57pG3Y+SMdKc06mp+eY/G5QcYUqb34pGTVqVL169YYOHXrjjTcmJSXJsvz888+//fbbY8aMad269ahRo+x2e6DtDc2uefyj+cu27ks7enj/1lU/f/7G6GbqI8NnHz3wyyNdQ8+vyzWDpu5JW3SXs+jH+MXH9rzdTXP+cbHpkyuPrJ/s0cLw2ceOzbzI34UUrp+598jq93p7fFAjNpj4x4G0eROTq7XTCzJO6m5wXjtt+4Ejx48fP340bd+2Vb98/eqdPeMwUx4hhBC6kISGo++9xpFXfNX4ezrUwneiaHir65+Y9ftPL1xfMQOYGEONbO839w5V3fDggjTZ3z6khuNnzpqQcHjGo/c/Pe94u+dmTRvZgAI39rx/8rB6O7546eEHJ31xIOymN2c919UAAACP/rpmydPtAjTEsxe9OHpEuVsfmZduK9i4eq9S7cPs0aPH77//3rNnT5PJNGPGDJPJZDKZpk+fXlZW1rNnz8WLF/fs2bPaO0cIIYQQQikpKd98841er3ctIYRwzl9++eUPPvig0mrA7jp27Dhs2LA777yzqKjI46GioqIxY8bceOONnTp1qnZXa1h02P3l+C+v76ziK1Tl0MKHRi6s9gqVbn4pKSsru+OOO9QxYTKZunTpohbcaNu2LWNs6dKlgVLZw3o9MXfWI6mnVi749KVd2RZjbIPmXVqHuB4m+vaPz3gvY9Sji0/6fZdyWaCR8TEaMeWWZ8fO3jLrgMO5lMfe8tT9rfT0aEwUwOmq7/WijJPaazQ0KkK7a9o9720sE0OiUrsNf+DF2Vc3mnjT6ytLamPvCCGEEKoM1/cZe0fijg/vWjx04Yvj+n26a0Uhdz40YMrcx65p3igxUscKM3ctnf/h+9/tKGYAANBr0qzHh7RvkRKpteesfvWmib8UuHYYP+r1V/qlz330ldT/vZTq3lJoeDg/fWTHwUOllXwrjmu6jxnbonD+iMlz9tlgxRZb01WfT7irzc9v7l319g3rHXYHAMCqLbY213zVsXNDYesBBQzhURH68lebvhoitpyje3Kcf6SMe+TGiG1TJ8/PqPZpg8TExFOnTj3wwAP//vuv+/J///134MCBM2fOTEpKqv7eEUIIIYT+q+6+++5XX33Ve7lOp/vpp58eeuih3bt3L1myZNy4cR9++GGQ+xw7duzMmTNLS0t9PlpaWjpz5sxx48bt3Lmzen2uUSpl+cvx+/+3rf7Icf0i3V8qC3FXTfzkp7V7DqUd2Ll6/pOd1de7UpspG4+5vtAntH5gyZG1z7YAgF7v7j625IHWzlfFJPKOHw/tfG9oWCvXCoE3BwCQUvo8/vmiVTsPHNy1buHbd7c1EAAgIU1vfe+HNXsOHUnbv/nvGSMudKU5RVEaNmxYr149URQTExNTU1PvvPPO2NhYACgsLPzpp5+GDRsmSZLZbPYfgNa3f+ydh5sc+PzOW+5/d+5vS5f9++v8We888fg3R863cnDT4S5vz3i4laGWuj3w5QV/rN6299DRo4d2rvrumTtvf+ajRau3Hzx0YPOfX0zsUH6hheTbv1q6df+hI4cPbFnyzZO9El1HQGM7j5/2w+rdBw8fPrhz3T8LPx3fGgD8XKPzaL24GJp9IqPhfY9eE1L+kLb1PQ9fV5p+gtWLjiKB2x348oI/Vm3dc+jo0UO7N/z8/oSORgLgMU4GTJn728qte9OOHjm4dcX8N+/tHO66B17adHjx+HjnnykPLDi04Z3uHm///B9y4MEpJHR7aMbP6/elpe3Z8OuHj/SIdm3nu88eCo/t2LFzx5a1yxa+//CTC04kjby1lwgAvSbN+nHZlr1pRw7vXffF8KjADfm7KF47CXh6Kx8VCCGE0BWGhF17183w69eL9y6evbRswNhbEs8/1qxX9+SjXzw1/u57H5+2xD7w2dlfP97K+Q9n22sGND711dP3jLnr4de/3lrovscz34zuffOzn2885/H1NxIZFelQNLHxEdrK+lS/c6fo4h0bD9gAAMC2bcMOltqxQzQAOKPPAEAjY+qJ1lMnzjIAgHcHtO31xq6gGiIRfR59uM/Zbz74NY9Vfn78WrBgQW5ubtOmTQ8dOuS+PC0trUmTJrm5uQsWLKjB7hFCCCGE/qNGjRrlc7nVal22bJlaK+PHH3+s0pzPXbp0WbduXYAV1q1b16VLlyr1011NAtABXo7rWzw6e/YjjbLmPDP+7vFPfjB/fYaaHO049uVdQwYPHjz4ppdWV0iX3rF2W1mjXt1j1b9Cr+rVhu5Yu9nsEdPyuzmAof3Tc2bdKm58b8KtY56bXTLwhS+eGxjCofkD/3utj+2Hx+64ecTY52b+vC+/BsdbHTqd7tdff/3www8XLVpkNpt79+598OBBq9UKACaT6b333jt8+LAoipRSfwWgua7v6JsTSv/8eOY+m99mTsye9PTf2okz3romolbCgE27dYk/9Mmj48bc9cSs/Y0mvvZcN+vityfdN/H5xY6uT3046Sq1EVa4a94Hk8aOHjHm0W/ONrr/o5euVXP+9S0emzP3ydaFP0x5YOzdE9/a6Ohwdack4vcanUcjoiJI2qzXf7D1f/D2liIAAI+5+aFbjWs/+XhNgTE6WhewXWjarUv84c+fvO+uux9990/l2udmPDvA6L5/gIBvFIPgt+mAg1PfatKcbx9MOTHr8XHjnvnsaJuJX3/9VDsx+D5X6IHFbOWiViuAj3e2ARryd1G8dxLw9FY+KhBCCKErC21w05heJb//sN4MlrXzf87tfvvIRm5fIeTm4xtWrd+4ftn3/3vgoc8zO9wzvnf5Q6zsyLqVm7ZtXbds6+mK/7gz5jOwq9GxwtKuT/+5ccfe7UtnPjowwf8L9eiYenAu95xzN8SWk11Io2Kj3XeWPOqtSa23ffbxsgKvVxaBG+Iptz5wA185fe4+h+eGVTJjxgxZlpOTkzMyKuRRnzp1qn79+rIsz5gxo0YNIIQQQgj9JzVs2NDfQ7Nnz+7evTsh5PDhww0aN
|
||
|
|
}
|
||
|
|
}
|
||
|
|
]
|
||
|
|
}
|
||
|
|
]
|