Probably you already tried Smoothing Edges with VRayEdgeTex, and you also experimented that sometimes this method does not work with imported models.

In this cases please verify the 3d model!
Many imported models have unwelded vertex: this is the problem.

VERIFY THE 3D MODEL

To check this add EDIT POLY modifier > select vertex > move a vertex.
Verify if the vertex moves without “envolve “the other vertex – image below:

SOLUTION:
In this case just fix this problem collapsing/welding the vertex and everything will be fine!