how to make a special boss that only appears once

You can make a global boolean variable in "variables tab", procedure in your dimension: When player enters dimension, then use procedure block: If: Get Global: [YOUR VARIABLE NAME] = false: Spawn at x: x y: y z: z living entity: [YOUR ENTITY], set Global: [YOUR VARIABLE NAME] to: true; and turn off spawning in your boss settings, so it will spawn only once. If you want it to spawn not right, where player spawns in the dimension, you can place numbers instead of XYZ and put there small procedure block, that has "0" in it by default, it's in "Math" tab at left side when creating procedure, there you can type any number, so these will be certain co-ordinates or use '[_] + [_] from the same tab, first will be "x" from the procedure block and second will be number procedure block, so you can make it spawn some block away from the player, for an example:

Spawn at x: x y: [Y + 100] z: z living entity: [YOUR ENTITY]

t will make it spawn 100 blocks above player spawn co-ordinates.

For making a custom portal to overworld spawn after boss dies:

You can make a new block, and make a procedure for "When entity collides in the block", and make procedure:

Choose "Switch dimension of [Event target entity] to: ["dimension name"]" and choose overworld(it's set by default).

You can make a procedure for "Custom entity dies" in your entity mod element, where:

Place [BLOCK] at x:x y:y z:z

and choose your new block in [BLOCK] part of the procedure block.