I find the separate-app approach to be much more convenient for the testers as it always leaves them a fallback option to the stable app, which is a whole separate app on their device. That way they will never be blocked out of using the app and might be more likely to try out the unstable version and help out with testing.
Build Type is the correct setting for this and it is enough to add the line
applicationIdSuffix ".beta"to the build type definition, which will modify the application id. This is required for publishing the effectively same app to the Play Store.