Idea for battle system (inspired by trading card games Hearthstone/MTG):
You have a selectable "deck" of abilities. Let's say you have a "deck" of 10 ability slots per Actor.
Each battle each Actor starts with 2 random abilities from your "deck" (not including escape or any other ability you want to always have).
Example: 3 Actors vs. 2 enemies
Actors and enemies get their starting abilities from their "decks" (6 abilities vs. 4 abilities)
Each turn for each Actor/enemy:
1) "draw" another ability.
2) Gain Resource (could be consistent like Hearthstone or random like MTG). OR no resource required (just RNG on what ability you receive, could do tier rated abilities for balance. Example: 1 Super strong ability, 2 strong abilities, 3 Medium abilities, 4 low abilities)
3) use ability(s) based on how many resources you have or with no resources required.
Can have random elements inserted to "draw" more abilities in a turn, etc. or make abilities that can draw more abilities. If using resource method, can make abilities to drain or steal resources from the enemy to essentially lock them out of a turn.
Recycle the abilities into deck once used or make deck bigger than 10 and once deck runs out for enemy or Actor they die.
CTB would probably be appropriate for turn order. However ATB could make use of counter abilities or instants (like in MTG).
Pros:
-Different take on turn-based RPG
-Near endless combinations of decks/strategies. Could make players constantly consider what's in their deck and if it's good enough for the dungeon ahead.
-Could make abilities found in chests or quests to expand their available abilities.
-Can incorporate mechanics like: Draw ability, extra turns.
-Each Actor should have their own playstyle/deck/abilities, so when you mix and match Actors you can get a variety of outcomes.
-Can still make abilities work with stats and leveling. Higher Str, more damage an ability does, etc.
Cons:
-Seems complicated to code (I have no coding experience so I don't even know if this is possible)
-Battles would probably be a lot slower, possibly consider fewer encounters or speed up the battle system somehow. Random encounters might not be a good idea for this. More of a DND/fixed encounter approach seems good where the players can sneak by the enemies if they chose to.
-Heavily relied on RNG, RNG can make every encounter seem brand new or can be completely miserable. Find a way to reduce the variance or give other consistent options (like maybe if you don't have the ability you want or resources to use an ability, then you can always: use item, escape, attack, etc.).
I don't plan on using this system in my project. This is just me up at 1am brain dumping lol.