Hello.
I used the “Get Cursor” API to get all records in my data(Japanese), but some times(4/10 times) text garbling problem occurred in the column value of the field.
Like this “数値_0” => “数��_0”
It’s not all records be like this. Only some records and every time that cause a problem will be the same record.
function GetCursor(hostname, api_token, payload, callback) {
const data = JSON.stringify(payload);
const options = {
hostname: hostname,
path: '/k/v1/records/cursor.json',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': lengthInUtf8Bytes(data),
'X-HTTP-Method-Override': 'GET',
'X-Cybozu-API-Token': api_token
}
}
Execute(options, data, callback);
}
This problem occurred only in kintone.com
In cybozu.com not cause this problem
Please check this problem cause by kintone API or my code and how to solve it.
thank you.
Hi Sarit,
There are some parts of the request that we do not have enough information to execute.
Could you check the following and try to isolate the problem?
Please check the following five things.
-
Does it happen with fields other than the number field?
-
Is the field code “数値_0” an automatically assigned field code?
-
If you replace the field code “数値_0” in the form settings, and then enter it manually again, does the same phenomenon occur?
-
Launch the developer tools in your web browser and execute the following process on the console screen
◆ First time
var body = {
'app': kintone.app.getId(),
'size': 500,
' __REQUEST_TOKEN__': kintone.getRequestToken()
};
var url = 'https://{subdomain}.kintone.com/k/v1/records/cursor.json';
var xhr = new XMLHttpRequest();
xhr.open('POST', url);
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.setRequestHeader('X-Cybozu-API-Token', '{api_token}');
xhr.onload = function() {
if (xhr.status === 200) {
// success
console.log(JSON.parse(xhr.responseText));
} else {
// error
console.log(JSON.parse(xhr.responseText));
}
};
xhr.send(JSON.stringify(body));
◆ Second time
var param = '{the id obtained on the first run}';
var url = 'https://{subdomain}.kintone.com/k/v1/records/cursor.json?id=' + param;
var xhr = new XMLHttpRequest();
xhr.open('GET', url);
xhr.setRequestHeader('X-Cybozu-API-Token', '{api_token}');
xhr.onload = function() {
if (xhr.status === 200) {
// success
console.log(JSON.parse(xhr.responseText));
} else {
// error
console.log(JSON.parse(xhr.responseText));
}
};
xhr.send(JSON.stringify(body));
- Same as 4. Please let us know if the phenomenon can be reproduced when processing on the console screen,
and if so, please provide the details of the process.
Best,
Sean
Thanks for your reply.
-
Yes, it also happened with the Text field in value and.
-
Yes, “数値_0” is an automatically assigned field code. All data imported by CSV file.
-
Yes, the same phenomenon occurs.
4,5. I repeated about 10 times, but the phenomenon can not be reproduced on the console screen.
Hello Sarit!
Since you can’t reproduce the error on the console, it might be due to the API environment or the execution methods.
But since it’s happening only on certain records, I’m not quite sure about that either…
Plus, because the error is occurring only on Kintone.com, the problem may be on the Kintone side.
Just to make sure, is it possible for you to send the “whole” script.
Thanks,
Chris
Thanks for your reply.
This is my whole script.
exports.getCursor = functions.https.onRequest((req, res) => {
cors(req, res, () => {
const host = req.body.host;
const token = req.body.token;
const payload = req.body.payload;
GetCursor(host, token, payload, function(data) {
res.send(data);
});
});
});
function GetCursor(hostname, api_token, payload, callback) {
const data = JSON.stringify(payload);
const options = {
hostname: hostname,
path: '/k/v1/records/cursor.json',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': lengthInUtf8Bytes(data),
'X-HTTP-Method-Override': 'GET',
'X-Cybozu-API-Token': api_token
}
}
Execute(options, data, callback);
}
function Execute(options, data, callback) {
const req = https.request(options, res => {
var str = '';
res.on('data', function(chunk) {
str += chunk;
});
res.on('end', function() {
obj = JSON.parse(str);
callback(obj);
})
})
req.on('error', error => {
console.error('err:' + error)
})
req.write(data)
req.end()
}
function lengthInUtf8Bytes(str) {
// Matches only the 10.. bytes that are non-initial characters in a multi-byte sequence.
var m = encodeURIComponent(str).match(/%[89ABab]/g);
return str.length + (m ? m.length : 0);
}
And my service that returns text garbling from the above script.
public async GetCursor(appEntity: AppEntity, id: string): Promise<any> {
return new Promise((resolutionFunc, rejectionFunc) => {
const body = {
host: appEntity.appDef.hostname,
token: appEntity.appDef.token,
payload: {
id: id
}
};
this.http.post<any>(this.functionUrl + "/getCursor", body,{headers:this.headers,responseType:"json"}).subscribe(data => {
let records = data['records'] as [];
let next = data['next'] as [];
if (records != null && next != null) {
resolutionFunc(data);
} else {
rejectionFunc(data);
}
});
});
}
Thank you.
Hello Sarit,
Thanks for the script.
Just in case, I contacted the Kintone team, and it seems like they haven’t seen any case of an error that happens only on the one side like this case.
And the script seems overall okay, but the “lengthInUtf8Bytes” function part seems suspicious to me. Can you try to debug this part to confirm whether it is properly encoded, comparing the pre-encoded text and post-encoded text?
Best,
Chris
Thanks for your reply.
lengthInUtf8Bytes function uses to get the length of app id.
pre-encoded : “ca6dd352-466a-4f71-b31a-36d7d2d1b576”
post-encoded: 36
Hi Sarit,
I had a couple more questions to ask and also clarification.
The text garbling may be caused due to the effect of the header specification.
Either the specification is incorrect, or the Content-Length is too short.
Also, since it is a Japanese text garbling, could you try and set the field code to half-width characters,
and see if the same phenomenon is reproduced?
Lastly, which web service are you using?