GRAFT: Gain-Recalibrated Adapters for Transformer-Based Neural Population Activity Modeling