An output style is a referencing style. EndNote output style files have rules and templates that set out how the in-text citations and bibliography will appear in your document. There are a number of styles available with the installation of EndNote. The name of the style refers to the journal or publisher that has defined the bibliographic format. Output styles can be modified and new styles can be created using EndNote's style editor.
This video is for EndNote 20. In EndNote x9 the Output Styles box is in the toolbar. The Style can also be chosen from the Output Styles submenu of the Edit menu.