airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard
verified publisher icon airbytehq

Source Google PageSpeed Insights - PatternSyntaxException

Open reidab opened this issue 3 years ago • 1 comments

Environment

  • Airbyte version: 0.40.26
  • OS Version / Instance: Docker image
  • Deployment: k8s on GKE via official Helm chart
  • Source Connector and version: Google PageSpeed Insights 0.1.0
  • Destination Connector and version: n/a
  • Step where error happened: Setup new source or connection

Current Behavior

  1. Fill out form
  2. Connection tests run and I see "All connection tests passed!"
  3. The call to /api/v1/sources/create to create the source fails with a PatternSyntaxException

Expected Behavior

No failures

Logs

{
    "message": "Internal Server Error: java.util.regex.PatternSyntaxException: Unclosed character class near index 87\n^(?:origin:)?(http(s)?:\\/\\/)[\\w.-]+(?:\\.[\\w\\.-]+)+[\\w\\-\\._~:\\/?#[\\]@!\\$&'\\(\\)\\*\\+,;=.]+$\n                                                                                       ^",
    "exceptionClassName": "com.networknt.schema.JsonSchemaException",
    "exceptionStack": [
        "com.networknt.schema.JsonSchemaException: java.util.regex.PatternSyntaxException: Unclosed character class near index 87",
        "^(?:origin:)?(http(s)?:\\/\\/)[\\w.-]+(?:\\.[\\w\\.-]+)+[\\w\\-\\._~:\\/?#[\\]@!\\$&'\\(\\)\\*\\+,;=.]+$",
        "                                                                                       ^",
        "\tat com.networknt.schema.JsonMetaSchema.newValidator(JsonMetaSchema.java:290)",
        "\tat com.networknt.schema.ValidationContext.newValidator(ValidationContext.java:57)",
        "\tat com.networknt.schema.JsonSchema.read(JsonSchema.java:203)",
        "\tat com.networknt.schema.JsonSchema.getValidators(JsonSchema.java:479)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:278)",
        "\tat com.networknt.schema.ItemsValidator.doValidate(ItemsValidator.java:99)",
        "\tat com.networknt.schema.ItemsValidator.validate(ItemsValidator.java:86)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:279)",
        "\tat com.networknt.schema.PropertiesValidator.validate(PropertiesValidator.java:69)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:279)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:262)",
        "\tat io.airbyte.validation.json.JsonSchemaValidator.validateInternal(JsonSchemaValidator.java:123)",
        "\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:137)",
        "\tat io.airbyte.server.handlers.SourceHandler.validateSource(SourceHandler.java:275)",
        "\tat io.airbyte.server.handlers.SourceHandler.createSource(SourceHandler.java:88)",
        "\tat io.airbyte.server.apis.SourceApiController.lambda$createSource$3(SourceApiController.java:48)",
        "\tat io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:18)",
        "\tat io.airbyte.server.apis.SourceApiController.createSource(SourceApiController.java:48)",
        "\tat java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)",
        "\tat java.base/java.lang.reflect.Method.invoke(Method.java:578)",
        "\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)",
        "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)",
        "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)",
        "\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)",
        "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)",
        "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)",
        "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)",
        "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)",
        "\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)",
        "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)",
        "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)",
        "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)",
        "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)",
        "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)",
        "\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)",
        "\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)",
        "\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)",
        "\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)",
        "\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)",
        "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)",
        "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)",
        "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)",
        "\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)",
        "\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)",
        "\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)",
        "\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)",
        "\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)",
        "\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)",
        "\tat org.eclipse.jetty.server.Server.handle(Server.java:501)",
        "\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)",
        "\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)",
        "\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)",
        "\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)",
        "\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)",
        "\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)",
        "\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)",
        "\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)",
        "\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)",
        "\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)",
        "\tat java.base/java.lang.Thread.run(Thread.java:1589)",
        "Caused by: java.util.regex.PatternSyntaxException: Unclosed character class near index 87",
        "^(?:origin:)?(http(s)?:\\/\\/)[\\w.-]+(?:\\.[\\w\\.-]+)+[\\w\\-\\._~:\\/?#[\\]@!\\$&'\\(\\)\\*\\+,;=.]+$",
        "                                                                                       ^",
        "\tat java.base/java.util.regex.Pattern.error(Pattern.java:2045)",
        "\tat java.base/java.util.regex.Pattern.clazz(Pattern.java:2709)",
        "\tat java.base/java.util.regex.Pattern.sequence(Pattern.java:2156)",
        "\tat java.base/java.util.regex.Pattern.expr(Pattern.java:2086)",
        "\tat java.base/java.util.regex.Pattern.compile(Pattern.java:1798)",
        "\tat java.base/java.util.regex.Pattern.<init>(Pattern.java:1445)",
        "\tat java.base/java.util.regex.Pattern.compile(Pattern.java:1084)",
        "\tat com.networknt.schema.PatternValidator$PatternValidatorJava.<init>(PatternValidator.java:76)",
        "\tat com.networknt.schema.PatternValidator.<init>(PatternValidator.java:45)",
        "\tat java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)",
        "\tat java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)",
        "\tat java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)",
        "\tat com.networknt.schema.ValidatorTypeCode.newValidator(ValidatorTypeCode.java:158)",
        "\tat com.networknt.schema.JsonMetaSchema.newValidator(JsonMetaSchema.java:283)",
        "\t... 67 more"
    ],
    "rootCauseExceptionClassName": "java.lang.Class",
    "rootCauseExceptionStack": [
        "java.util.regex.PatternSyntaxException: Unclosed character class near index 87",
        "^(?:origin:)?(http(s)?:\\/\\/)[\\w.-]+(?:\\.[\\w\\.-]+)+[\\w\\-\\._~:\\/?#[\\]@!\\$&'\\(\\)\\*\\+,;=.]+$",
        "                                                                                       ^",
        "\tat java.base/java.util.regex.Pattern.error(Pattern.java:2045)",
        "\tat java.base/java.util.regex.Pattern.clazz(Pattern.java:2709)",
        "\tat java.base/java.util.regex.Pattern.sequence(Pattern.java:2156)",
        "\tat java.base/java.util.regex.Pattern.expr(Pattern.java:2086)",
        "\tat java.base/java.util.regex.Pattern.compile(Pattern.java:1798)",
        "\tat java.base/java.util.regex.Pattern.<init>(Pattern.java:1445)",
        "\tat java.base/java.util.regex.Pattern.compile(Pattern.java:1084)",
        "\tat com.networknt.schema.PatternValidator$PatternValidatorJava.<init>(PatternValidator.java:76)",
        "\tat com.networknt.schema.PatternValidator.<init>(PatternValidator.java:45)",
        "\tat java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)",
        "\tat java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)",
        "\tat java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)",
        "\tat com.networknt.schema.ValidatorTypeCode.newValidator(ValidatorTypeCode.java:158)",
        "\tat com.networknt.schema.JsonMetaSchema.newValidator(JsonMetaSchema.java:283)",
        "\tat com.networknt.schema.ValidationContext.newValidator(ValidationContext.java:57)",
        "\tat com.networknt.schema.JsonSchema.read(JsonSchema.java:203)",
        "\tat com.networknt.schema.JsonSchema.getValidators(JsonSchema.java:479)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:278)",
        "\tat com.networknt.schema.ItemsValidator.doValidate(ItemsValidator.java:99)",
        "\tat com.networknt.schema.ItemsValidator.validate(ItemsValidator.java:86)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:279)",
        "\tat com.networknt.schema.PropertiesValidator.validate(PropertiesValidator.java:69)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:279)",
        "\tat com.networknt.schema.JsonSchema.validate(JsonSchema.java:262)",
        "\tat io.airbyte.validation.json.JsonSchemaValidator.validateInternal(JsonSchemaValidator.java:123)",
        "\tat io.airbyte.validation.json.JsonSchemaValidator.ensure(JsonSchemaValidator.java:137)",
        "\tat io.airbyte.server.handlers.SourceHandler.validateSource(SourceHandler.java:275)",
        "\tat io.airbyte.server.handlers.SourceHandler.createSource(SourceHandler.java:88)",
        "\tat io.airbyte.server.apis.SourceApiController.lambda$createSource$3(SourceApiController.java:48)",
        "\tat io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:18)",
        "\tat io.airbyte.server.apis.SourceApiController.createSource(SourceApiController.java:48)",
        "\tat java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)",
        "\tat java.base/java.lang.reflect.Method.invoke(Method.java:578)",
        "\tat org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)",
        "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)",
        "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)",
        "\tat org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)",
        "\tat org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)",
        "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)",
        "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)",
        "\tat org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)",
        "\tat org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)",
        "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)",
        "\tat org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)",
        "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:292)",
        "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:274)",
        "\tat org.glassfish.jersey.internal.Errors.process(Errors.java:244)",
        "\tat org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)",
        "\tat org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)",
        "\tat org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)",
        "\tat org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)",
        "\tat org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)",
        "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)",
        "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)",
        "\tat org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)",
        "\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)",
        "\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)",
        "\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)",
        "\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)",
        "\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)",
        "\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)",
        "\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)",
        "\tat org.eclipse.jetty.server.Server.handle(Server.java:501)",
        "\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)",
        "\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)",
        "\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)",
        "\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)",
        "\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)",
        "\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)",
        "\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)",
        "\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)",
        "\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)",
        "\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)",
        "\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)",
        "\tat java.base/java.lang.Thread.run(Thread.java:1589)"
    ]
}

Steps to Reproduce

  1. Fill out form to set up a Google PageSpeed Insights connection
  2. Connection tests run and I see "All connection tests passed!"
  3. The call to /api/v1/sources/create to create the source fails with a PatternSyntaxException

Are you willing to submit a PR?

Yes

reidab avatar Feb 02 '23 01:02 reidab

Thanks for the PR, our team will take a look!

sajarin avatar Feb 02 '23 13:02 sajarin

Any chance of a review on this soon? I'd really like to try using this connector to automate some internal workflows, but this issue means that I can't successfully configure it at all.

I'd build and deploy a custom container, but I'm not entirely clear on how that should be accomplished since this change needs to propagate into the generated source specs file in addition to the connector container itself.

reidab avatar Feb 15 '23 08:02 reidab

@sajarin Any word on this? As near as I can tell, this connector is unusable without this fix.

reidab avatar Mar 10 '23 00:03 reidab

@sajarin @reidab I'm also looking to use this connector and ran into the same error. Any update on when the fix for this (#22287) might be merged? It looks like it got the pr-alpha-beta label last week, but I'm not sure what the process is on this repo

tylerdigital avatar Apr 25 '23 05:04 tylerdigital