To "unsplit" editor groupswithout closing any open files, use the menu-bars: View > Editor Layout > Single (or, while in the editor, press and release ALT and then type vls).
Editor groups are also closed by default when they become empty. You can empty an editor group by closing tabs/files or by dragging all of the tabs from that group to another.
Hints:
Grabbing the area to the right of the tabs allows you to drag/drop all tabs in a group at once.
The "Open Editors" section of the "Explorer" panel lets you use CTRL and SHIFT to select and drag/close multiple editors at once. (If you don't have an "Open Editors" section, use the "..." menu of the Explorer panel to enable it.)
You can open new terminal groups by pressing the '+' icon in the terminal tab of the panel.
"Splitting" a terminal (done by clicking the split icon in the terminal tab of the panel) really just opens an additional terminal in the same group and displays it side-by-side with the others in the group.
So, two ways to "unsplit" terminals:
Kill terminals (e.g. with the trashcan icon) until there is only one left in the group.
Switch to a new terminal group with only a single terminal in it.
To avoid this "VSCode irritation no. 1", you also have, beside "View: Join Editor Group with Next Group", a way to avoid the issue entirely, with VSCode 1.52 (Nov. 2020)
A new setting workbench.editor.splitOnDragAndDrop allows to disable editor group splitting when using drag & drop of editors.
There is also a way to toggle this conditionally during the drag and drop operation when you press and hold Shift key (macOS) or Alt key (Windows, Linux).