Why don't my reported file and storage sizes match the sizes reported on my storage platform or an online converter such as Google?
DataHub stores file sizes in base-2 quantities. However, these values are displayed with base-10, or SI (International System of Units), prefixes (for example, megabytes and gigabytes). Those prefixes are used since most users are familiar with that notation. Because of the risk of confusing the user with the lesser-known base-2 notation (for example, mebibytes or gibibytes), we chose to display base-10 notation. See the screenshot on the right for an example.
When using a conversion tool such as searching "808884388 bytes to MB" in Google, be sure to either specify "MiB" (mebibytes) in the search string or change the resulting units to an appropriate base-2 unit of measure.
I'm seeing "No such host is known" errors in the job log, but I know the platform is accessible from the machine on which DataHub is installed. Why am I seeing this and how can I fix it?
"No such host is known" errors are caused by DNS resolution failures. Most of the time DataHub can recover from these errors and retry the transfer successfully. However, if there are permanent issues with DNS resolution, DataHub will not be able to resolve the host until these are addressed.
The first step in fixing these issues is to review the job log to determine which URLs are failing. Start by searching for
"Failure": "No such host is known" in the job log file. The corresponding
RequestUri value will provide the failing URL. Then use the appropriate networking tools such as
nslookup to troubleshoot and fix the DNS resolution.
Please keep in mind that the same host name may resolve to different IP addresses based on the geographic location of the server. Any required network changes will need to be made in relation to the server(s) on which DataHub is installed. Also note that many platforms will redirect traffic to other domains or sub-domains based on the request. For example, Box file uploads will be redirected to
upload.box.com and file downloads to
dl.boxcloud.com. Firewall rules must account for these subtle differences in order for DataHub to successfully run jobs.