500 error when attempting to connect to index

[toddcoulson]

I have the following code, but I keep getting a 500 error whenever I send an index call to the client. I cannot figure out where my issue is:

require_once "../vendor/autoload.php";
use Aws\ElasticsearchService\ElasticsearchPhpHandler;
use Elasticsearch\ClientBuilder;
$handler = new ElasticsearchPhpHandler('us-west-2');
$client = ClientBuilder::create()
    ->setHandler($handler)
    ->setHosts(['https://search-arc-bis-XXXXX.us-west-2.es.amazonaws.com:443'])
    ->build();
$params = array();
$params['body']  = array(
    'name' => 'Todd Coulson'
);
$params['index'] = 'item';
$params['type']  = 'item_books';
$client->index($params);

If I comment out the last line, no error comes up. It seems as though it could be a elasticsearch-php issue, but I cannot figure out where the issue is coming in. I am authenticating through the .aws/credentials file. Can you help me troubleshoot this?

[jeskew]

Do you get the same error if you send calls other than index to the client?

[toddcoulson]

Do you have an example. I found this code on elasticsearch-php and inserted it and it seemed to work:

$params = array();
$params = [
    'index' => 'test',
    'type' => 'test',
    'id' => 1,
    'client' => [
        'future' => 'lazy'
    ]
];
$future = $client->get($params);

Also, why would index not work, could it possibly be an issue with elasticsearch? Sorry in advance for any newb questions, this is my first foray into elasticsearch and aws. I am also in the process of setting up my domain again. The above code seemed to work without a domain set up on aws (which is weird). I would expect your part of it to fail whenever credentials do not work. Anyway, that domain should be set up soon, so I can test again then. EDIT: ok, I got a domain now. Tested both index and search calls off $client and both give a 500 error. I went back to that future = lazy code and it worked. I am still not sure what code from elasticsearch-php relies on the credentials and why I keep getting 500 error. It's also hard to get any results back telling me what part of my code may be wrong.

[toddcoulson]

I just ran the following calls in the terminal to make sure I can read and write to my aws account and it works: Last login: Mon Sep 11 11:15:16 on ttys004

$ curl -XPUT https://search-arc-bis-XXX.us-west-2.es.amazonaws.com/movies/movie/1 -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}'
{"_index":"movies","_type":"movie","_id":"1","_version":1,"result":"created","_shTTTodToToddsToTodTodToToToToTTTTTTTTTT
$ curl -XGET 'search-arc-bis-XXX.us-west-2.es.amazonaws.com/ms/_search?q=mars'
{"took":19,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0}

[jeskew]

If sending an unsigned request with curl succeeds, are you able to send a request successfully after removing the signing handler altogether? If so, then the error is likely in this library; if not, then the error could be in elasticsearch-php or in the Elasticsearch cluster.

Also, do you have any more debugging information about the 500 error encountered? Is the service returning any kind of message?

[toddcoulson]

I attached what is coming back in the header for the 500 error. I need more clarification of "removing the signing handler altogether"? Does that mean taking out the .aws file? Because that is how I am getting authentication at the moment.

EDIT: is the curl just an api for CRUD? Doesn't seem like it has to do with credentials at all. I just ran a delete call and it worked, after I moved the .aws credentials. My .aws is similar to this:

[default]
aws_access_key_id = AKIAXXXXXXXUQ
aws_secret_access_key = I8EqGJ7dDu6fXXXXXX/kB
[bis]
aws_access_key_id = AKIAXXXXXXXUQ
aws_secret_access_key = I8EqGJ7dDu6fXXXXXX/kB

both are the same account. Do you have a recommended way of getting credentials with your tool?

[jeskew]

Ah, didn't realize the 500 was coming from your application and not from your elasricsearch service endpoint. Could you take a look at your server logs to find more informations on the error response you're receiving from the elasticsearch endpoint? The original HTTP status code and error message would go a long way to solving the issue.

By "removing the signing handler altogether, I meant to comment out the call to setHandler on the ClientBuilder instance. curl is a library for sending HTTP requests that (as far as I know) does not perform any AWS SigV4 signing. The fact that you can successfully curl on the elasticsearch endpoint indicates that it is not using IAM authentication, so it would be helpful to see if you encounter any issues when using elasticsearch-php's default HTTP handler.

[toddcoulson]

I cannot get any logs from nginx. I am also running this inside a docker container. I looked in a few places where the logs might be, but keeps telling me

tail: unrecognized file system type 0x794c7630... please report this to [email protected]. reverting to polling

Also commenting out setHandler does not give a 500 error, it just hangs not fully loading the page - Gateway Timeout. TIA for all your help.