Trying to restore CouchDB from one server to another, but not able to reindex it using the below command.
RAILS_ENV=production bundle exec rake sunspot:reindex
after reindexing 5 out of 7 batches, the rake is aborted with the error ArgumentError: invalid date
I have attached a screenshot for reference.
Dear @sanket252, which Primero implementation are you supporting including which version and country?
Hi @childprotectioninnov, thanks for the responce.
We are using Primero IMS version 1.6.20 in India.
Dear @sanket252 are you running a live/production version of Primero? If not, could you try installing and running Primero v2 instead of v1.6 since we have a more up-to-date software with regular releases with v2?
@childprotectioninnov Yes, we are running a production instance of Primero v1.6.20
The issue is with the date format. Make sure they are in UTC format. From the docs
date field shall be of the form 1995-12-31T23:59:59Z The trailing “Z” designates UTC time and is mandatory Optional fractional seconds are allowed, as long as they do not end in a trailing 0 (but any precision beyond milliseconds will be ignored).
How do we find bad date data from the DB?
Maybe try RAILS_ENV=production bundle exec rake sunspot:reindex --trace ?
RAILS_ENV=production bundle exec rake sunspot:reindex --trace
This just fails with the error mentioned in this issue 
Did you try stopping and then starting solr?
Either use rake stop/start or just kill the PID. You could also try removing the solr folder after stopping and before starting and it should regenerate (not tested so do at your own risk)
I did restart the actual server and then tried to reindex using
RAILS_ENV=production bundle exec rake sunspot:reindex --trace
It continues to fail. Is there a way to get more information on where it’s failing?
The current error is not that useful.
rake aborted!
ArgumentError: invalid date
The only thing I can think of (apart from upgrading) is grepping through the database looking for dates without the Z at the end. So a value of “2020-10-10 23:59:59+0300” should be expressed as “2020-10-10T20:59:59Z” for solr
Thank you and apologies for the delayed response. I was offline for a few days.
I am checking data in the DB and most of the date fields are of the following format: ‘2022/03/31 00:00:00 +0000’
Re indexing is working for some data but fails for few.
Any thoughts.
Can you find out which dates it works/fails on? I guess some dates managed to get into the db in a form solr does not like so you need to find and edit them. It should be pretty obvious I think which they are
I am trying to find the record or date which is resulting in failure. However, not making much progress.
Error info is limited. Is there a possibility to get a more detailed error?
We have around 3K records with lots of date fields. So struggling a bit to find out the error record.
Any help is appreciated.
The problem was with the error handling in sunspot package.
There was a bad date entry empty space for one of the records that failed when Date.parse() tried to parse the date.
I had to debug the sunspot code to figure out the field that resulted in this issue. After correcting the record reindexing worked.
That’s very cool you fixed it!
So you looked in the db for empty space in date fields? If so we can add this more generally to the main documentation
Sorry, for the delayed response. Yes, I did.
What was the command you ran?