Connectors and External Integrations

This page documents how SureDMS connects to external systems through its connector and integration layer. It covers the SureConnect microservices platform, the Angular frontend services, the connector lifecycle, and how external repositories are associated with a SureDrive or study.

Architecture Overview

The connector system spans three layers that work together:

Layer	Location	Responsibility
Microservices integration	SC/SureConnect/	Spring Cloud stream connectors to external platforms
Frontend integration UI	suredms-web-client JS integration/	Configuration dialogs, repository listing, document migration flows
Desktop connector runtime	suredms-desktop-client-connector	Java client bridging the desktop client to Nuxeo and local XML archives

The frontend and web service layers handle real-time REST-based interactions. Bulk or scheduled data transfers to external systems (Medidata, Veeva, etc.) go through SureConnect streams.

---

SureConnect — Microservices Layer

Located at SC/SureConnect/, this is a Spring Cloud Data Flow integration platform that handles asynchronous data streams between SureDMS and external systems.

Supported Connector Types

The connectors/http/ folder defines stream source starters for each platform:

Starter	External System
spring-cloud-starter-stream-source-medidata	Medidata Rave (clinical trial data)
spring-cloud-starter-stream-source-veeva	Veeva Vault
spring-cloud-starter-stream-source-ms365	Microsoft 365 / SharePoint
spring-cloud-starter-stream-source-google	Google Workspace (GSuite)
spring-cloud-starter-stream-source-boomi	Boomi (Dell / integration bus)
spring-cloud-starter-stream-source-medrio	Medrio EDC
spring-cloud-starter-stream-source-mulesoft	MuleSoft
spring-cloud-starter-stream-source-snaplogic	SnapLogic

Transport Protocols

Transport adapters sit alongside HTTP starters under connectors/:

• connectors/ftp/ — FTP file transfer
• connectors/sftp/ — Secure FTP
• connectors/jdbc/ — Database / JDBC source
• connectors/file/ — File system source
• connectors/time/ / connectors/trigger/ — Scheduled and event triggers
• connectors/transform/ — In-stream data transformation

Orchestration

spring-cloud-dataflow/ manages stream pipeline orchestration and deployment. spring-cloud-dataflow-ui/ provides the management UI for those pipelines.

---

Frontend Integration Layer

Integration Types Registered

IntegrationConfigurationService in SC/suredms-web-client/src/main/webapp/app/js/integration/integration-configuration.js maps every integration type constant to its own configuration service:

Constant	Type	Configuration Service
INTEGRATION_TYPE_CMIS	CMIS / Content Management	IntegrationConfigurationContentManagementService
INTEGRATION_TYPE_CUSTOMER	Customer	IntegrationConfigurationCustomerService
INTEGRATION_TYPE_BOOMI	Dell Boomi	IntegrationConfigurationDellService
INTEGRATION_TYPE_EDC	Electronic Data Capture	IntegrationConfigurationElectronicDataCaptureService
INTEGRATION_TYPE_GOOGLE	Google Workspace	IntegrationConfigurationGoogleService
INTEGRATION_TYPE_MEDIDATA	Medidata Rave	IntegrationConfigurationMedidataService
INTEGRATION_TYPE_MEDRIO	Medrio	IntegrationConfigurationMedrioService
INTEGRATION_TYPE_SFTP	Secure FTP	IntegrationConfigurationSecureFtpService
INTEGRATION_TYPE_MS365	Microsoft 365	IntegrationConfigurationMicrosoftService
INTEGRATION_TYPE_SURE_API	Sure API	IntegrationConfigurationSureApiService
INTEGRATION_TYPE_SURE_DRIVE	SureDrive-to-SureDrive	IntegrationConfigurationSureDriveService
INTEGRATION_TYPE_VEEVA	Veeva Vault	IntegrationConfigurationVeevaService
INTEGRATION_TYPE_CASTOR	Castor EDC	IntegrationConfigurationCastorService
INTEGRATION_TYPE_DATATRAK	Datatrak	IntegrationConfigurationDatatrakService

Each configuration service opens its own dialog, reads server properties via ServerPropertyRepository, and can invoke a connection test before saving.

Key Frontend Files

File	Angular Service / Component	Purpose
integration/integration-service.js	IntegrationService	MS365 settings cache, user-integration mapping, calendar history, OAuth state
integration/integration-configuration.js	IntegrationConfigurationService	Routes configureIntegration() calls to the correct per-type service
integration/integration-manager.js	integrationManager component	Lists all integrations, connects/disconnects, opens config dialogs, enriches connector status
integration/integrations-study-repository.js	IntegrationStudyRepositoryService, integrationsStudyRepository component	Loads repository list for a study, enriches each repository with type icon and connector status
integration/integrations-study-mapper.js	StudyDocumentMapperController	Maps external source folders to SureClinical content types for a specific drive
integration/integrations-study-mapping.js	Study mapping view	Shows saved folder-to-content-type mappings
integration/integrations-study-clone-folders.js	StudyCloneFoldersController	Clones external folder structure into the SureDrive during migration
integration/integration-configuration-sure-drive.js	IntegrationConfigurationSureDriveService	Opens the SureDrive connector configuration dialog; loads properties from INTEGRATION_TYPE_SURE_DRIVE
administration/integrations/integration-repositories.js	ConnectorStatusService	Provides enrichStatus() to annotate repository objects with live connector health state
administration/integrations/integrations-dashboard.js	Admin dashboard controller	Displays all integrations system-wide via IntegrationService.getIntegrations()

Connector Status

ConnectorStatusService.enrichStatus(repository.status) is called on every repository object loaded — in IntegrationStudyRepositoryService, StudyDocumentsMigrationController, integrationManager, and network-contacts. It annotates status objects with display-ready properties (icon, color, label) based on the live connector state.

Platform-Specific Sub-Folders

• integration/ms365/ — MS365 OAuth admin component (integrations-ms365-auth-admin.js)
• integration/google/ — Google OAuth flow
• integration/dropbox/ — Dropbox connector
• integration/apple/ — Apple integration
• integration/suregpt/ — SureGPT / AI provider configuration (IntegrationsSureGPTService)
• integration/classification/ — Document classification driven by integration type
• integration/cloud-resource-upload.js — Cloud resource upload handler

---

Route Registration

Integration routes are registered in SC/suredms-web-client/src/main/webapp/app/js/state/app-states.js under the app.network.study.browse state hierarchy:

State	URL Pattern	Template / Controller
app.network.study.browse.repository	/repository	<integrations-study-repository> component
app.network.study.browse.documents-migration-mapper	/migration-mapper?importProcessId&entityId&repositoryId&integrationType&mappingId&readOnly&autoMapping	integrations-study-mapper.html / StudyDocumentMapperController
app.network.study.browse.mappings	/mappings?entityId&repositoryId&integrationType&mappingId&readOnly	integrations-study-mapping.html
app.network.study.browse.clone-folders	/clone-folders?importProcessId&quickStartInit	integrations-study-clone-folders.html / StudyCloneFoldersController
app.monitoring.integrations	/integrations?page	monitoring-integrations.html / MonitoringIntegrationsController

The documents-migration, documents-migration-repositories, documents-migration-mappings, and documents-migration-preview states handle the step-by-step document migration wizard that connects an external repository source to a SureDrive destination.

---

Connector Lifecycle

1. System-Level Configuration (Admin)

An admin navigates to the integrations administration area. IntegrationConfigurationService.configureIntegration() dispatches to the per-type service (e.g., IntegrationConfigurationMedidataService). Each service opens a dialog that:

• Reads current settings from ServerPropertyRepository.getIntegrationsServerProperties(integrationType).
• Allows editing credentials, endpoint URLs, tenant IDs, and feature flags.
• Exposes a Test Connection button that calls IntegrationsRepository to verify credentials against the external system before saving.
• Persists changes via ServerPropertyRepository.updateServerProperty().

Settings are stored as keyed ServerProperty entries (e.g., integrations.ms365.enabled, integrations.ms365.calendarHistory).

2. Enabling a Connector

integrationManager.connectToIntegration() confirms with the user, then sets integration.propertyEnabled = true via ServerPropertyRepository.updateServerProperty(). On success, ConnectorStatusService.enrichStatus() updates the displayed state in the integration tile.

3. User-Level Authentication (OAuth)

For MS365 and Google, a user-specific OAuth link is required. The <integrations-ms365-auth-admin> component (integration/ms365/integrations-ms365-auth-admin.js) loads tenant settings via IntegrationService.loadSettings(INTEGRATION_TYPE_MS365.id) and checks tenant alignment before triggering the authorization flow via UserIntegrationsRepository.

IntegrationService caches the user's integration mapping in service.ms365.userIntegrationMapping so the rest of the UI can gate calendar and contact import features behind IntegrationService.isMS365Enabled() and IntegrationService.getUserIntegrationMapping().

4. Associating a Repository with a SureDrive/Study

The Repositories tab in a SureDrive (app.network.study.browse.repository) renders the <integrations-study-repository> component. On init, IntegrationStudyRepositoryService.loadRepositories() calls IntegrationsRepository.getIntegrationRepository() with moduleFeature: INTEGRATION_FEATURE_DOCUMENT_MANAGEMENT to fetch all repositories linked to the current study. Each repository object is enriched with:

• repository.type — resolved integration type constant.
• repository.icon — derived from the integration module or type.
• repository.status — enriched by ConnectorStatusService.enrichStatus().

5. Document Migration / Mapping

Clicking through the migration wizard takes a user across these states in order:

1. Repository selection — documents-migration-repositories: pick the external system.
2. Mapping overview — documents-migration-mappings: review or create folder-to-content-type mappings.
3. Detailed mapper — documents-migration-mapper / StudyDocumentMapperController: drag-and-drop folder-to-content-type assignment per external repository folder. Accepts integrationType, repositoryId, mappingId as state parameters.
4. Clone folders — clone-folders / StudyCloneFoldersController: copies the external folder structure into the SureDrive.
5. Preview — documents-migration-preview: reviews what will be imported before committing.

6. Sync and Data Import

• MS365 calendar sync: IntegrationService.importCalendarEvents(startDate, autoSync, callback) triggers an import into the Quality Calendar or Management Calendar. Export direction uses IntegrationService.exportCalendarEvents().
• Contact import: NetworkContactsController checks IntegrationService.getUserIntegrationMapping() before surfacing the MS365 import option.
• Bulk data (Medidata, Veeva): StudyDocumentsMigrationController drives the migration process with connector status tracking via ConnectorStatusService.

7. Monitoring

app.monitoring.integrations → MonitoringIntegrationsController provides a system-wide integration health view. ConnectorStatusService.enrichStatus() is the shared status normalization point used across all views.

---

SureDrive-to-SureDrive Connector

INTEGRATION_TYPE_SURE_DRIVE (IntegrationConfigurationSureDriveService) covers drive-to-drive connector scenarios. The configuration dialog loads server properties via ServerPropertyRepository.getIntegrationsServerProperties(INTEGRATION_TYPE_SURE_DRIVE.id). The implementation notes in the source (// todo: missing implementation) indicate this connector type is partially scaffolded and not yet fully wired.

---

Desktop Connector Path

The desktop client does not use the SureConnect stream layer. It communicates with the backend via NuxeoClient in suredms-desktop-client-connector. See Desktop Connector and Runtime for full details on the desktop-side connector architecture.

---

Data Flow

User / Admin (Angular UI)
       │
       ▼
IntegrationConfigurationService
       │ configureIntegration()
       ▼
Per-Type Config Service (e.g., IntegrationConfigurationMedidataService)
       │ getIntegrationsServerProperties / updateServerProperty
       ▼
ServerPropertyRepository → REST API → suredms-web-service
       │
       ├──► Direct call (MS365 OAuth, Google OAuth, Castor API)
       │
       └──► SureConnect (Spring Cloud Data Flow)
                  │
                  ├── spring-cloud-starter-stream-source-medidata
                  ├── spring-cloud-starter-stream-source-veeva
                  ├── spring-cloud-starter-stream-source-ms365
                  └── spring-cloud-starter-stream-source-google
                             │
                             ▼
                  SureDrive / Study (mapped content types)
                  via StudyDocumentMapperController + IntegrationsRepository

---

Dependencies

Service	Used By
IntegrationsRepository	IntegrationStudyRepositoryService, integrationManager, migration controllers
ServerPropertyRepository	All per-type configuration services, IntegrationService
UserIntegrationsRepository	IntegrationService (MS365 user mapping), MS365 auth admin
ConnectorStatusService	integrationManager, IntegrationStudyRepositoryService, StudyDocumentsMigrationController, NetworkContactsController
CacheRefreshService	IntegrationStudyRepositoryService (cache invalidation after repository changes)