I'm no expert in creating formulas unfortunately and I'm not quite sure what exactly was wrong with the formula apart from the numbers being a bit off.
Aside the final damage numbers there are differences how changing the default ATK/DEF would affect the formula result, for example:
If you use a formula in the form [
X * a.atk - b.def] then DEF will reduce the damage by a fixed amount. That means that attackers with very low ATK will probably do little damage or will not be able to break through your defense at all, while attackers with very high ATK will probably do almost the same damage as they would have done against a defenseless target.
That is again measured in "attacks required to defeat the target", in reality this might be a bit more complex.
If you use a formula in the form [
X * a.atk / b.def] then DEF will reduce the damage by a fixed percentage. That means debuffing a 500 DEF target by 50% would have a similar effect as debuffing a 100 DEF target. In addition a high ATK attacker hitting a very low DEF target will probably do many times the damage they'd do against an appropriate target.
---
If you just want to match your numbers, one suggestion would still be to use the base formula from my last post and factor the targets DEF in
(so an average DEF value does not change the preferred outcome):
(0.1 * Math.pow(a.atk, 1.75)) * ([average DEF value] / b.def)
For example, if you balance your formula around DEF being ATK / 2 you could replace the placeholder with that. In this case that would increase the total exponent for your ATK to 2.75, which means a 50% DEF buff would be negated by a ~16% ATK buff from the attacker.
---
To make a formula match certain damage values you'd usually choose multiple variables that do not affect the features of your formula you need
(make sure the variables can't be combined). Depending on the variables, there might not be a solution or one you absolutely don't like sometimes
(like a negative modifier for your ATK).
For example, if you want your first example to exactly result in 35 damage and your second one to result in 5000 damage you could try to change the two constants in your damage modifier to achieve that:
(16 + 12 - 6) * (a + (1 + 12) / b) = 35,
(200 + 116 - 100) * (a + (65 + 116) / b) = 5000
Tools like
Wolfram Alpha can also help solving equations like this:
a = -7705 / 99792 (≈ -0.077)
b = 99792 / 12805 (≈ 7.793)
If you prefer flat numbers, you could round these results to
a = 0 and
b = 8. In this case you would still get
35.75 and
4887 damage respectively.
(This particular solution might not be perfect of course since removing the "1 +" probably means that very low ATK values might result in 0 damage, even when they surpass the targets DEF)
I don't know if I understood what you were looking for exactly, but I still hope any of this helps.
:D