It looks like you're using the rock ledge tile, which has a different set of rules for passability than a normal floor tile does. However, if you're in map-editing mode with the pencil tool selected, try right-clicking the space below stairs B and C (this will select the tile that's currently placed there), then shift-left clicking in the same spot. That should make it passable from all directions, allowing you to go up onto the stairs. I *think* they should already be accessible from the tile above them, but if not, do the same thing there.
If you do have to do the space above each stair, you'll notice that it removes the edge to the top of the tile above stair B - you can fix that by going up one more space, then right-clicking it (to select the ground tile), and doing a normal left click in the same space (which will put the ledge back into place on the top of the tile without removing the passability you added on the bottom by shift-left clicking).
